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SUMMARY 


The  validity  of  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  was  demonstrated 
for  the  Air  Force's  Job  Performance  Measurement  System  (JPMS)  for  seven  of  eight  Air  Force 
specialties.  Although  the  validity  of  the  ASVAB  was  modest  in  comparison  to  its  validity  for 
training  school  grades,  the  results  indicated  that  the  ASVAB  does  predict  hands-on  job 
performance. 


PREFACE 


This  work  was  conducted  in  partial  fulfillment  of  Contract  No.  F41 689-86-D-0052  awarded 
to  Universal  Energy  Systems  Incorporated,  with  the  Air  Force  Human  Resources  Laboratory 
(AFHRL).  It  complements  the  AFHRL  Training  Systems  Division  efforts  in  job  performance 
criterion  development  by  investigating  the  structure  of  the  Air  Force’s  job  performance 
measurement  system  and  the  predictability  of  the  Armed  Services  Vocational  Aptitude  Battery 
for  that  system. 
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STRUCTURE  OF  THE  AIR  FORCE’S  JOB  PERFORMANCE 
MEASUREMENT  SYSTEM  AND  PREDICTABILITY  OF  THE 
ARMED  SERVICES  VOCATIONAL  APTITUDE  BATTERY  (ASVAB) 


I.  INTRODUCTION 

A  major  goal  of  the  Air  Force’s  Job  Performance  Measurement  project  is  to  develop  a 
measurement  system  to  use  for  validating  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB).  The  validity  evidence  for  using  the  ASVAB  to  select  and  classify  incumbents  into 
Air  Force  specialties  has  been  developed  previously  from  training  performance  criteria  (e.g., 
Wilbourn,  Valentine,  &  Ree,  1984).  These  training  criteria  were  the  grades  that  students 
received  to  reflect  their  success  in  technical  training  prior  to  being  assigned  to  their  initial, 
specialty  position.  Unfortunately,  training  school  grades  are  an  indication  of  the  potential  for 
job  performance  and  only  indirectly  relate  to  actual  on-the-job  performance. 

A  Congressional  mandate  directed  the  Armed  Services  in  1980  to  establish  research  programs 
to  validate  the  ASVAB  directly  against  job  performance  criteria  (Wigdor  &  Green,  1986).  This 
mandate  recognized  the  indirect  nature  of  the  validity  evidence  previously  accumulated  for  the 
ASVAB.  In  response  to  this  mandate,  the  Air  Force  Human  Resources  Laboratory  initiated 
systematic  research  on  several  methods  and  procedures  to  develop  a  job  performance 
measurement  system  (JPMS). 

An  initial  version  of  the  measurement  system  was  developed  and  applied  to  four  Air  Force 
specialties  (cf.  Hedge  &  Teachout,  1986):  Air  Traffic  Control  Operator  (AFS  272X0),  Avionic 
Communications  (AFS  328X0),  Information  Systems  Radio  Operator  (AFS  492X1),  and  Jet 
Engine  Mechanic  (AFS  426X2).  Subsequently,  the  measurement  system  was  expanded  to 
include  job  knowledge  tests  of  technical  proficiency.  This  expanded  system  was  applied  to 
four  additional  specialties:  Aerospace  Ground  Equipment  Mechanic  (AFS  423X5),  Aircrew  Life 
Support  (AFS  122X0),  Personnel  (AFS  732X0),  and  Precision  Measurement  Equipment  Laboratory 
(AFS  324X0). 

The  purpose  of  the  present  research  effort  is  to  describe  results  for  the  application  of  the 
Air  Force’s  JPMS  to  the  eight  specialties.  The  technical  approach  required  assessment  of 
the:  (a)  conceptual  structure  of  the  job  performance  measures  and  (b)  predictability  of  the 
ASVAB  for  the  job  performance  measures. 


Conceptual  Structure  of  the  Job  Performance  Measures 


Job  performance  is  a  complex  concept,  consisting  of  several  dimensions  that  are  predicted 
by  many  human  attributes.  This  complexity  requires  multiple  measures  of  job  performance 
that  are  homogeneous  in  content  and  relatively  independent  of  each  other  (Dunnette,  1963: 
Guion,  1976).  Such  multiple  measures  clarify  the  conception  of  job  performance,  and  thereby, 
enhance  the  understanding  of  predictors  of  job  performance. 

The  Air  Force’s  JPMS  emphasizes  the  classification  of  job  performance.  The  broadest 
classification  defines  the  content  of  job  performance  to  reflect  either  the  (a)  technical  or  (b) 
interpersonal  aspects  of  work  (Kavanagh,  Borman,  Hedge,  &  Gould,  1986).  At  the  next  level 
of  classification,  the  technical  and  interpersonal  components  are  classified  by  dimensions 
(Dickinson,  Hedge,  &  Ballentine,  1987).  Each  dimension  still  reflects  technical  or  interpersonal 
performance.  However,  the  conception  of  performance  is  enriched  by  the  delineation  of  technical 
and  interpersonal  performance  into  dimensions. 
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One  type  of  dimensional  classification  emphasizes  a  task-oriented  description  of  job 
performance.  It  relies  on  the  job  analyses  developed  and  maintained  by  the  Air  Force’s 
Occupational  Measurement  Center.  These  job  analyses  are  used  to  sample  tasks  in  order  to 
develop  a  relevant  and  measurable  description  of  job  performance. 

The  task-oriented  dimensions  are  subclassified  by  tasks.  Each  dimension  is  described  by 
a  set  of  interrelated  tasks  that  reflect  the  content  of  that  dimension.  Further,  the  tasks  can 
be  also  subclassified  into  steps  that  reflect  the  elemental  or  "correct-  versus  "incorrect"  aspects 
of  task  performance. 

A  second  type  of  dimensional  classification  emphasizes  a  trait-oriented  description  of  job 
performance.  The  trait-oriented  description  emphasizes  aspects  of  job  performance  that  are 
relatively  independent  of  work  content,  but  still  important  to  job  success.  This  descnpuc.i 
draws  on  organizational  norms  that  reflect  the  qualities  and  standards  appropriate  for  measuring 
each  member’s  contribution  to  the  Air  Force. 

The  Air  Force’s  JPMS  also  emphasizes  the  usage  of  multiple  methods  and  sources  for  job 
performance  information  (Kavanagh  et  al.,  1986).  Methods  are  the  procedures  for  obtaining 
information,  and  they  include  testing  and  rating.  Sources  are  the  individuals  who  provide  the 
information,  and  they  include  test  administrators,  supervisors,  incumbents,  and  peers.  The  Air 
Force  uses  test  administrators  to  collect  information  with  work  sample  and  job  knowledge  tests, 
and  it  uses  supervisors,  incumbents,  and  peers  to  obtain  information  with  rating  procedures. 


Armed  Services  Vocational  Aptitude  Test  Battery 


The  Armed  Services  began  uniform  testing  in  1950  with  the  use  of  the  Armed  Forces 
Qualification  Test  (AFQT).  The  AFQT  was  used  as  a  measure  of  general  trainability  to  set 
standards  for  entry  into  the  Armed  Services  (U.S.  Department  of  Defense,  1985).  A  sample 
of  officers  and  enlisted  men,  who  served  in  the  military  during  1944,  was  used  to  establish 
the  percentile  scores  for  setting  enlistment  standards.  These  scores  were  utilized  until  October 
1984  to  make  enlistment  decisions. 

The  ASVAB  (i.e.,  Forms  6  and  7)  was  first  used  in  1976  by  all  of  the  Armed  Services  for 
selection  and  classification.  Prior  to  that  date,  earlier  forms  of  the  ASVAB  were  used  for  (a) 
high  school  testing  programs  by  the  Armed  Services,  (b)  selection  and  classification  only  by 
the  Air  Force  and  Marine  Corps,  or  (c)  experimental  purposes.  Subsequent  to  uniform  ASVAB 
testing  by  the  Armed  Services,  additional  forms  have  been  developed.  Forms  8,  9,  and  10 
were  used  from  1980  through  1984.  Those  forms  appearing  after  1984  are  parallel  to  Forms 
8,  9,  and  10.  Currently,  Forms  14,  15,  16,  and  17  are  in  operational  use. 

In  1980,  a  nationally  representative  sample  of  young  people  was  administered  ASVAB  Form 
8a  (U.S.  Department  of  Defense,  1982).  The  Form  8a  scores  of  this  representative  sample 
were  used  to  develop  new  percentile  norms  for  ASVAB  Forms  8  and  above.  The  new  norms 
were  used  to  set  enlistment  standards  after  October  1984. 


II.  METHOD 


Participants 


Job  performance  data  were  collected  between  the  years  1985  and  1987  on  1,613  first-term 
(i.e.,  1  to  48  months  of  total  active  military  service)  job  incumbents  from  the  eight  specialties. 
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These  job  incumbents  were  stationed  at  63  US  Air  Force  bases  located  in  the  United  States, 
Great  Britain,  Europe,  and  the  Philippines. 

Sample  sizes  varied  between  the  specialties  because  of  the  number  of  personnel  assigned 
to  each  specialty  and  the  convenience  of  testing  available  personnel.  Further,  sample  sizes 
varied  within  each  specialty  according  to  the  job  performance  measure  due  to  missing  observations 
or  incomplete  data.  Nonetheless,  the  maximum  sample  sizes  were  as  follows:  Air  Traffic 
Control  Operator  (N  =  191),  Avionic  Communications  (N  =  98),  Information  Systems  Radio 
Operator  (N  =  157),  Jet  Engine  Mechanic  (N  =  255),  Aerospace  Ground  Equipment  Mechanic 
(N  a  261),  Aircrew  Life  Support  (N  =  191),  Personnel  (N  =  197),  and  Precision  Measurement 
Equipment  Laboratory  (N  =  138). 


Job  Performance  Measures 

Job  performance  information  was  obtained  with  testing  and  rating  procedures.  The  testing 
procedures  provided  information  on  technical  proficiency,  while  the  rating  procedures  provided 
information  on  technical  and  interpersonal  proficiency. 


Walk-Through  Performance  Testing 


One  testing  procedure  employs  work  sample  tests  and  is  referred  to  as  Walk-Through 
Performance  Testing  (WTPT).1  This  procedure  includes  traditional  hands-on  performance  tests 
that  are  administered  by  trained  personnel  (Hedge  &  Lipscomb,  1987).  For  example,  a  hands-on 
test  for  a  jet  engine  mechanic  requires  the  incumbent  to  install  a  starter  on  a  jet  engine.  As 
the  starter  is  installed,  the  test  administrator  uses  a  checklist  to  indicate  whether  each  task 
step  is  performed  correctly.  The  WTPT  procedure  also  includes  interview  performance  tests 
that  are  administered  systematically  by  trained  personnel.  For  example,  an  interview  test  for 
a  jet  engine  mechanic  requires  the  incumbent  to  explain  the  step-by-step  procedures  that  must 
be  employed  to  install  a  starter  on  a  jet  engine.  This  verbal  explanation  is  done  at  the  work 
site  such  that  the  explanation  can  be  supplemented  with  gestures  to  appropriate  tools  or 
equipment  in  order  to  clarify  the  procedures.  The  distinction  between  the  hands-on  and 
interview  tests  is  clear.  A  hands-on  test  emphasizes  “can  do"  the  task,  while  an  interview 
test  emphasizes  "knows  how  to  do." 

The  use  of  interview  tests  was  required,  because  many  tasks  cannot  be  measured  with 
hands-on  tests  due  to  safety,  time,  or  cost  constraints.  In  addition,  the  interview  testing 
procedure  was  seen  as  a  potential  substitute  or  surrogate  for  the  more  expensive  hands-on 
procedure. 

The  number  of  tasks  measured  with  WTPT  ranged  from  15  (i.e.,  Jet  Engine  Mechanics, 
AFS  426X2)  to  26  (i.e.,  Aerospace  Ground  Equipment  Mechanics,  AFS  423X5).  The  hands-on 
and  interview  tasks  required  from  less  than  a  minute  to  45  minutes  to  complete.  In  total, 
WTPT  required  from  3  to  7  hours  to  complete,  depending  on  specialty. 

Performance  for  some  tasks  was  measured  only  with  a  hands-on  or  interview  procedure, 
while  performance  for  other  tasks  was  measured  with  both  of  the  procedures  (Hedge,  Teachout, 


'WTPT  may  refer  interchangeably  to  Walk-Through  Performance  Testing,  the  process,  or  to  Walk-Through  Performance 
Test,  the  instrument. 
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&  Laue,  1990).  Thus,  several  performance  scores  were  available  from  WTPT.  In  the  present 
research,  the  total  scores  for  unique  hands-on  and  interview  tests  were  used  for  analysis.  In 
addition,  a  total  WTPT  score  was  used,  and  it  was  calculated  as  the  sum  of  scores  for  all 
hands-on  tests  plus  those  scores  for  the  unique  interview  tests. 

The  development  of  the  WTPT  measures  relied  heavily  on  input  from  subject  matter  experts 
(SMEs)  at  each  phase  of  the  process.  The  first  phase  in  WTPT  development  involved  task 
selection.  For  each  specialty,  the  current  Occupational  Survey  Reports  and  information  on  the 
technical  school  curricula  were  used  initially  to  screen  tasks  that  were  performed  by  first-term 
airmen.  Tasks  were  selected  that  were  (a)  performed  by  at  least  30%  of  the  first-term  airmen, 
(b)  included  in  the  technical  school  curricula,  and  (c)  most  critical  to  first-term  airman  performance. 
Additional  tasks  were  selected  that  were  associated  with  specific  equipment  (e.g.,  type  of  jet 
engine)  or  workcenter  (e.g.,  engine  repair  in  the  shop  or  on  the  flightline).  The  result  of  task 
selection  was  the  identification  of  (a)  a  primary  list  of  20  to  30  tasks  that  described  major 
aspects  of  a  first-term  airman’s  job  performance  and  (b)  an  alternate  list  of  tasks  to  be  used 
for  replacement  of  primary  tasks  eliminated  later  in  WTPT  development. 

The  second  phase  of  WTPT  development  required  validating  the  task  selection.  Over  a 
period  of  four  to  five  days,  SMEs  met  in  a  workshop  to  assess  the  appropriateness  of  the 
selected  tasks.  Tasks  from  the  primary  iist  were  eliminated,  because  they  were  obsolete, 

similar  in  performance  requirements  to  other  tasks,  not  routinely  performed,  or  difficult  to 

observe.  When  SMEs  eliminated  a  task,  a  parallel  task  from  the  alternative  list  was  chosen 
randomly  as  a  replacement.  The  final  product  of  this  workshop  was  a  list  of  tasks  that  was 
used  not  only  for  the  development  of  WTPT  measures,  but  also  for  the  development  of  job 
knowledge  and  rating  measures. 

The  third  phase  of  WTPT  development  required  task  analysis  to  determine  for  each  task 
the  (a)  suitability  for  hands-on  and  interview  testing  (e.g.,  availability  of  equipment),  (b) 
step-by-step  performance  requirements,  and  (c)  tools  and  equipment  configuration  needed  for 
testing.  For  each  specialty,  task  analysis  was  conducted  by  interview  and  observation.  Visits 
were  made  to  various  Air  Force  bases  to  meet  with  SMEs  and  to  observe  task  performance 
by  first-term  airmen. 

Next,  WTPT  tests  were  constructed  for  each  of  the  tasks.  The  mix  of  the  tests  was 

targeted  to  be  50%  of  the  tasks  measured  only  with  the  hands-on  procedure,  25%  only  with 

the  interview  procedure,  and  25%  with  both. 

The  fifth  phase  of  WTPT  development  required  validating  the  test  construction.  A  group 
of  SMEs  met  in  a  workshop  to  review  and  revise  the  tests.  Tests  were  examined  for  the 
clarity  of  instructions  and  the  validity  of  task  performance  steps.  In  addition,  the  tests  were 
sequenced  for  their  order  of  administration. 

After  this  workshop,  the  hands-on  and  interview  measures  were  pilot  tested  in  a  field 
setting.  Procedures,  directions,  performance  steps,  and  time  limits  were  examined,  and  if 
appropriate,  revisions  were  made  to  the  tests. 

The  seventh  phase  of  WTPT  development  was  a  test  scoring  workshop  that  was  conducted 
with  SMEs  to  determine  the  relative  importance  and  criticality  of  each  test  and  its  performance 
steps.  This  information  was  used  to  develop  weighting  schemes  for  each  test  as  well  as  a 
weighting  scheme  for  the  total  WTPT  score. 

Finally,  test  administrators  were  selected  and  trained.  All  test  administrators  were  required 
to  be  either  active  duty  or  recently  retired  SMEs  to  ensure  their  technical  expertise  and 
familiarity  with  the  Air  Force.  The  training  was  extensive  and  focused  on  practice  in  administering, 
evaluating,  and  scoring  the  hands-on  and  interview  tests.  Videotape  materials  were  used  for 
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practice  and  to  illustrate  correct  and  incorrect  step-level  performance.  Further,  WTPT  measures 
were  administered  in  Dretest,  field  settings  to  provide  a  "dress  rehearsal"  prior  to  full-scale 
data  collection. 


Job  Knowledge  Measures 


These  measures  were  paper-and-pencil  tests  that  were  developed  simultaneously  with  the 
hands-on  and  interview  tests  (Bentley,  Ringenbach,  &  Augustin,  1989).  The  content  of  the  job 
knowledge  tests  corresponded  closely  to  that  of  the  WTPT  measures,  since  the  job  knowledge 
tests  were  seen  as  potential  surrogates  for  the  WTPT  measures.  Sets  of  items  were  written 
for  a  job  knowledge  test  to  measure  the  tasks  that  were  covered  by  the  WTPT  measures. 
The  items  were  written  and  reviewed  by  several  groups  of  SMEs  in  workshop  contexts.  The 
job  knowledge  test  was  administered  in  the  pretest,  field  setting  along  with  the  WTPT  measures 
to  gather  data  for  item  analysis.  These  data  were  used  to  eliminate  poor  items  and  reduce 
test  length  to  meet  time  constraints. 


Rating  Measures 


Ratings  were  obtained  for  all  of  the  classifications  of  job  performance  from  supervisors, 
incumbents,  and  peers.  Many  of  the  decisions  that  guided  the  format  and  content  of  the 
rating  measures  were  made  in  the  developmental  efforts  associated  with  the  Jet  Engine  Mechanic 
specialty  (AFS  426X2).  This  was  the  first  specialty  for  which  the  JPMS  was  developed. 

The  task  rating  form  provided  the  most  specific  rating  data.  This  rating  form  contained 
task  statements  for  all  of  the  tasks  measured  by  the  hands-on  and  interview  tests,  plus  tasks 
that  were  eliminated  during  the  WTPT  developmental  process  due  to  time  or  logistical  constraints. 
The  task  statements  were  rated  with  5-point  scales  anchored  with  adjectives  on  each  point. 

The  task  ratings  only  for  the  task  statements  corresponding  to  the  hands-on  and  interview 
tests  were  used  in  the  present  research.  These  ratings  were  averaged  to  obtain  an  indication 
of  technical  proficiency  at  the  task  level  of  classification. 

The  task-oriented  dimensional  rating  form  required  supervisors,  incumbents,  and  peers  to 
rate  technical  proficiency  on  task-oriented  dimensions.  Potential  dimensions  were  identified 
through  factor  analysis  of  occupational  survey  data  in  a  series  of  workshops  with  SMEs,  the 
dimension  definitions  and  representative  tasks  were  discussed,  and  5-point  rating  scales  were 

constructed  for  each  dimension.  Behavioral  descriptions  for  each  of  the  five  points  were 

developed  using  the  behavioral  summary  statement  approach  advocated  by  Borman  (1979). 

The  dimension  scores  were  averaged  to  obtain  an  indication  of  technical  proficiency  using 
task-oriented  dimensions. 

The  trait-oriented  dimensional  rating  form  was  developed  to  be  representative  of  all  specialties 
in  the  Air  Force.  It  focuses  on  eight  traits  that  distinguish  effective  performers  across  all  jobs. 
This  Air  Force  wide  form  was  constructed  by  a  group  of  ‘resource  managers"  in  the  Air  Force 
who  had  managerial  responsibilities  for  a  large  number  of  specialties.  They  were  able  in 
discussion  to  (a)  compare  the  performance  requirements  of  several  specialties  and  (b)  reach 
consensus  on  an  inter-specialty  perspective  of  performance.  In  addition,  the  manage'-?  ^ycloped 
5-point  rating  scales  that  were  anchored  with  behavioral  summary  statements.  The  dimensions 
were:  (a)  technical  knowledge/skill;  (b)  initiative/effort;  (c)  knowledge  of  and  adherence  to 

regulations/orders;  (d)  integrity;  (e)  leadership;  (f)  military  appearance;  (g)  self-development; 
and  (h)  self-control.  For  the  trait-oriented  rating  form,  the  technical  knowledge/skill  dimension 
was  used  to  indicate  technical  proficiency,  while  the  remaining  dimension  scores  were  averaged 
to  indicate  interpersonal  proficiency. 
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The  global  rating  form  was  developed  to  measure  technical  and  interpersonal  proficiencies 
needed  for  successful  performance.  The  two  items  were  also  developed  in  a  workshop  setting. 
The  two  types  of  proficiency  were  discussed  and  defined,  and  the  behavioral  summary  approach 
was  used  to  place  specific  behavioral  descriptions  on  5-point  scales. 


ASVAB  Measures 


The  ASVAB  scores  for  the  participants  were  obtained  from  their  personnel  records.  These 
scores  included  values  for  the  ten  ASVAB  subtests,  the  AFQT,  and  the  four  Air  Force  composites 
(i.e.,  Mechanical,  Administrative,  General,  and  Electronics)  used  to  classify  personnel  to  a 
specialty. 

Four  of  the  ASVAB  subtests  (i.e.,  Arithmetic  Reasoning,  Numerical  Operations  (half  weighted). 
Paragraph  Comprehension,  and  Word  Knowledge)  are  combined  to  form  the  AFOT. 

The  Air  Force's  composites  for  classification  are  formed  from  combinations  of  the  ten  ASVAB 
subtests  as  follows:  Mechanical  (Mechanical  Comprehension,  General  Science,  Auto/Shop 
Information  (double  weighted)),  Administrative  (Numerical  Operations,  Coding  Speed,  Word 
Knowledge,  Paragraph  Comprehension),  General  (Word  Knowledge,  Paragraph  Comprehension, 
Arithmetic  Reasoning),  and  Electronics  (Arithmetic  Reasoning,  Mathematics  Knowledge,  Electronics 
Information,  General  Science). 

The  eight  specialties  examined  in  this  research  were  distributed  uniformly  across  the  four 
classification  composites.  Thus,  the  Mechanical  composite  included  Jet  Engine  Mechanic  (AFS 
426X2)  and  Aerospace  Ground  Equipment  Mechanic  (AFS  423X5)  specialties;  the  Administrative 
included  Information  Systems  Radio  Operator  (AFS  492X1)  and  Personnel  (AFS  732X0);  the 
General  included  Air  Traffic  Control  Operator  (AFS  272X0)  and  Aircrew  Life  Support  (AFS 
122X0);  and  Electronics  included  Avionic  Communications  (AFS  328X0)  and  Precision 
Measurement  Equipment  Laboratory  (AFS  324X0). 

Participants  with  incomplete  ASVAB  data,  and  those  tested  with  Forms  5,  6,  or  7  were 
eliminated  from  the  samples.  Since  Forms  5,  6,  and  7  are  normed  to  the  1944  reference 
population,  they  could  not  be  compared  to  Forms  8  and  above  which  are  normed  to  the  1980 
reference  population.  Test  scores  from  Forms  8  and  above  were  transformed  to  a  common 
metric  using  available  equating  formulas. 


Final  Training  School  Grades 


Final  grades  received  in  technical  training  school  were  also  available  from  the  personnel 
records.  Since  final  training  school  grades  have  typically  been  used  to  describe  the  validity 
of  ASVAB  predictors,  they  served  as  a  comparison  criterion  for  the  JPMS  measures.  These 
grades  ranged  from  70  to  100,  with  70  being  the  minimum  passing  grade.  Approximately  four 
percent  of  the  students  enrolled  in  technical  school  in  the  Air  Force  fail  to  receive  a  passing 
grade  (Wilbourn  et  al.,  1984). 
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III.  RESULTS 


Assessment  of  Job  Performance  Measures 


The  focus  of  these  analyses  is  a  construct-oriented  evaluation  of  the  job  performance 
measures  for  each  of  the  eight  specialties.  Such  a  focus  yields  information  necessary  to 
judge  the  appropriateness  of  each  performance  measure  for  validating  the  ASVAB.  The  analysis 
approach  included  factor  analyses  of  the  measures  to  assess  the  Air  Force’s  conception  of 
job  performance  as  well  as  a  comparison  of  the  structures  obtained  from  the  eight  specialties. 
In  addition,  the  rating  forms  were  used  in  regression  analyses  to  predict  performance  on 
measures  from  WTPT.  These  regression  analyses  determined  whether  the  less  expensive 
rating  forms  could  serve  as  substitutes  for  the  WTPT  measures.  Finally,  the  rating  forms 
themselves  were  compared  to  assess  supervisors,  incumbents,  and  peers  as  sources  of  rating. 
The  comparisons  required  (a)  analyses  of  variance  with  post  hoc  comparisons  to  determine 
mean  differences  among  the  rater  sources  and  (b)  multitrait-multimethod  analyses  to  determine 
the  amount  of  method  bias  due  to  rating  sources  as  well  as  the  validity  of  trait-  and  task-oriented 
dimensions  for  describing  job  performance. 


Factor  Analyses 


The  Air  Force’s  conception  of  job  performance  logically  leads  to  an  expected  factor  structure. 
For  each  specialty,  two  general  performance  factors  were  expected  to  reflect  technical  and 

interpersonal  proficiencies.  In  addition,  four  method  factors  were  expected  that  reflected  the 
type  of  measurement  procedure.  One  method  factor  was  that  of  testing,  as  reflected  by  WTPT 
and  job  knowledge  measures.  Method  factors  were  also  expected  for  the  rating  measures. 
Based  on  rating  source  research  (e.g.,  Holzbach,  1978;  Klimoski  &  London,  1974;  Lawler,  1967, 
1975;  Tsui  &  Ohlott,  1988),  a  separate  method  factor  was  expected  fo  supervisory,  self-,  and 
peer  ratings. 

The  maximum  likelihood  procedure  was  used  to  extract  factors;  this  procedure  tends  to 
produce  the  most  accurate  estimates  of  population  parameters  (Gorsuch,  1974).  The  scree 
test  and  number  of  eigenvalues  greater  than  1.00  were  used  as  criteria  to  determine  the 

number  of  factors  (Gorsuch,  1974).  Using  these  criteria,  only  five  factors  were  extracted  for 
each  specialty;  the  sixth  (i.e.,  testing  method)  factor  was  not  obtained  for  any  specialty. 
Subsequently,  the  extracted  factors  were  rotated  analytically  using  oblimin  with  delta  specified 
to  equal  zero  (Jennrich  &  Sampson,  1966;  Norusis,  1986).  The  nature  of  these  five  factors 
and  their  intercorrelations  were  similar  across  the  specialties.  The  factor  titles  are  displayed 
in  Table  1.  The  factor  structure  and  factor  correlation  matrices  for  the  eight  specialties  are 
reported  in  Tables  A-1  to  A-16  in  Appendix  A. 

The  interpretation  of  the  factors  employed  general  rules.  First,  larger  loadings  were  inspected 
(e.g.,  an  absolute  value  of  .700  or  greater).  Such  loadings  accounted  for  49%  or  more  of  a 
measure’s  variance  and  were  likely  to  characterize  the  content  of  a  factor.  Second,  smaller 
loadings  were  inspected  (e.g.,  an  absolute  value  from  .500  up  to  .700).  These  loadings  could 

also  characterize  the  content  of  a  factor,  providing  that  the  measure  had  very  small  loadings 

on  the  remaining  factors  (e.g.,  -.300  to  .300).  Third,  the  pattern  of  loadings  on  a  factor  was 
used  for  interpretation.  For  example,  if  the  absolute  values  of  peer  measure  loadings  tended 
to  be  greater  than  those  of  other  types  of  measures,  the  factor  was  characterized  as  a  peer 
factor. 

Four  highly  similar  factors  were  found.  The  most  prominent  of  the  factors  was  a  technical 
proficiency  factor.  As  expected,  the  measures  that  characterized  the  technical  proficiency  factor 
were  the  hands-on  and  interview  components  of  WTPT  as  well  as  the  job  knowledge  measure. 


7 


Moreover,  inspection  of  the  loadings  in  Tables  A-1,  A-3,  A-5,  A-7,  A-9,  A-11,  A-13,  and  A-15 
reveals  that  rating  measures  which  reflected  technical  proficiency  loaded  higher  on  that  factor 
than  rating  measures  which  reflected  interpersonal  proficiency. 

The  three  remaining  prominent  factors  reflected  supervisory,  self,  and  peer  points  of  view. 
For  seven  of  the  eight  specialties,  the  supervisory  factor  was  characterized  by  high  loadings 
by  all  supervisory  measures,  regardless  of  technical  or  interpersonal  content.  For  all  specialties, 
the  self  factor  was  characterized  by  high  loadings  by  all  self-rating  measures.  For  six  of  the 
eight  specialties,  the  peer  factor  was  characterized  by  measures  with  high  loadings  by  all  peer 
measures. 


Table  1.  Titles  of  the  Factors  by  Specialty 


Factor 


Specialty 

1 

II 

III 

IV 

V 

AIRTRAFF 

TECH 

SUPER 

SELF 

PEER 

PEER-TEC 

AVIONIC 

TECH 

SUPER 

SELF 

PEER-TEC 

PEER-INP 

ISRADIO 

TECH 

SUP-TEC 

SELF 

PEER 

SUP-INP 

JETMECH 

TECH 

SUPER 

SELF 

PEER 

INPERS 

AEROSPACE 

TECH 

SUPER 

SELF 

PEER 

PEER-TEC 

AIRCREW 

TECH 

SUPER 

SELF 

PEER 

PEER-TEC 

PERSON 

TECH 

SUPER 

SELF 

PEER-TEC 

PEER-INP 

PRECISE 

TECH 

SUPER 

SELF 

PEER 

SELF-TEC 

Note  Abbreviations  for  the  specialties  are:  AIRTRAFF,  Air  Traffic  Control  Operator;  AVIONIC,  Avionic 

Communications;  ISRADIO,  Information  Systems  Radio  Operator;  JETMECH,  Jet  Engine  Mechanic;  AEROSPACE, 
Aerospace  Ground  Equipment  Mechanic;  AIRCREW,  Aircrew  Life  Support;  PERSON,  Personnel;  and  PRECISE,  Precision 
Measurement  Equipment  Laboratory.  Abbreviations  for  factors  are:  TECH,  proficiency  of  the  technical  aspects  of 
work;  SUPER,  overall  performance  from  supervisory  point  of  view;  SELF,  overall  performance  from  incumbent's  point 
of  view;  PEER,  overall  performance  from  a  peer  point  of  view;  PEER-TEC,  proficiency  on  technical  aspects  of  work 
from  peer  point  of  view;  SUP-TEC,  technical  proficiency  from  supervisory  point  of  view;  SUP-INP,  proficiency  on 
interpersonal  aspects  of  work  from  supervisory  point  of  view;  PEER-INP,  proficiency  on  interpersonal  aspects  of  work 
from  peer  point  of  view;  SELF-TEC,  proficiency  on  technical  aspects  of  work  from  incumbent  point  of  view  as  contrasted 
with  supervisor  and  peer  points  of  view;  and  INPERS,  overall  proficiency  on  interpersonal  aspects  of  work  from 
supervisory,  incumbent,  and  peer  points  of  view. 


The  supervisory  and  self  factors  had  small  positive  correlations  with  the  technical  proficiency 
factor  (i.e.,  .026  to  .383),  while  the  peer  factor  had  small  positive  and  negative  correlations 
(i.e.,  -.158  to  .361)  with  the  technical  proficiency  factor.  An  exception  to  these  patterns  occurred 
for  the  Aerospace  Ground  Equipment  Mechanic  specialty.  For  this  specialty,  all  factors  correlated 
negatively  (i.e.,  -.158  to  -.341)  with  the  technical  proficiency  factor. 

Further,  the  supervisory  and  self  factors  tended  to  correlate  positively  with  each  other,  while 
both  had  negative  and  positive  correlations  with  the  peer  factor. 

The  fifth  and  final  factor  was  complex  and  changed  in  its  nature  by  specialty.  This  factor 
was  characterized  with  loadings  by  peer,  supervisory,  and  interpersonal  measures.  The  Jet 
Engine  Mechanic  specialty  provided  the  most  interpretable  solution,  and  its  results  serve  as 
reference  for  the  remaining  specialty  results. 

As  shown  in  Table  A-7,  the  Jet  Engine  Mechanic  factor  structure  indicates  that  the  fifth 
factor  reflected  interpersonal  proficiency.  This  interpersonal  factor  is  characterized  by  moderate 
loadings  by  all  interpersonal  measures.  Further,  as  shown  in  Table  A-8,  this  factor  has  small 
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correlations  with  the  rater  source  factors  (i.e.,  supervisory,  self,  peer),  and  it  has  a  small 
negative  correlation  with  the  technical  proficiency  factor. 

The  Air  Traffic  Control  Operator  specialty  did  not  provide  an  interpersonal  proficiency  factor. 
As  shown  in  Table  A-1,  a  peer-technical  factor  was  obtained.  This  factor  was  characterized 
by  high  loadings  on  task  and  dimension  peer  ratings.  In  contrast  to  other  specialties,  the 
peer  and  peer-technical  factors  correlated  negatively  with  other  factors.  Apparently,  in  this 
specialty,  interpersonal  proficiency  was  not  a  distinct  aspect  of  performance. 

The  Avionic  Communications  specialty  was  characterized  by  two  peer  factors  that  represented 
technical  and  interpersonal  points  of  view.  Although  the  peer  technical  factor  had  moderate 
loadings  by  supervisory  and  self  measures  of  technical  proficiency,  this  factor  was  dominated 
by  the  peer  point  of  view  (see  Table  A-3).  The  peer  interpersonal  factor  had  moderate  loadings 
from  supervisory  measures  of  interpersonal  proficiency,  but  it  was  also  dominated  by  the  peer 
point  of  view.  As  shown  in  Table  A-4,  the  peer  technical  factor  had  the  largest  correlation 
(i.e.,  .328)  with  the  technical  proficiency  factor,  and  the  peer  interpersonal  had  the  smallest 
(i.e.,  .160). 

The  Information  Systems  Radio  Operator  specialty  provided  a  factor  that  reflected  a 
supervisory  point  of  view  of  interpersonal  proficiency.  As  shown  in  Table  A-5,  the  two  measures 
that  characterized  this  factor  were  supervisory  ratings  of  interpersonal  proficiency  on  the  Air 
Force  wide  and  global  forms.  For  this  specialty,  supervisory  ratings  were  separated  into 
technical  and  interpersonal  points  of  view.  Interestingly,  the  supervisory  technical  factor  had 
the  greater  correlation  with  the  technical  proficiency  factor  (i.e..  .383  versus  -.115). 

The  Aerospace  Ground  Equipment  Mechanic  and  Aircrew  Life  Support  specialties  haa  a 
fifth  factor  that  reflected  technical  proficiency  from  a  peer  point  of  view  (see  Tables  A-9  and 
A-1 1,  respectively).  This  factor  was  characterized  primarily  by  the  task  and  dimension  rating 
measures.  As  shown  in  Tables  A- 10  and  A- 12,  this  peer  technical  factor  correlated  negatively 
with  technical  proficiency  (i.e.,  -.389  and  -.315,  respectively). 

The  Personnel  specialty  had  two  peer  factors.  One  factor  reflected  technical  proficiency 
from  the  peer  point  of  view,  while  the  second  peer  factor  reflected  interpersonal  proficiency. 
As  shown  in  Table  A-14,  both  peer  factors  correlated  negatively  with  technical  proficiency  (i.e., 
-.273  and  -.130,  respectively). 

Finally,  the  Precision  Measurement  Equipment  Laboratory  specialty  had  a  fifth  factor  that 
reflected  technical  proficiency  from  the  incumbent  point  of  view  (see  Table  A-1 5).  As  shown 
in  Table  A-1 6,  this  factor  correlated  negatively  with  technical  proficiency  (i.e.,  -.232).  Interestingly, 
the  supervisory  and  peer  interpersonal  measures  loaded  modestly  on  this  factor  but  with  a 
sign  opposite  to  the  incumbent  technical  measures.  Thus,  for  this  factor,  incumbents  who 
rated  themselves  as  low  on  technical  proficiency  measures  were  likely  to  be  rated  as  high  on 
interpersonal  proficiency  measures  by  supervisors  and  peers. 


Prediction  of  WTPT  Measures  by  Rating  Form  Composites 


The  rating  forms  were  used  to  predict  WTPT  performance  for  each  specialty.  For  each 
rating  form,  multiple  regression  analyses  were  completed  using  three  composite  predictors. 
These  predictors  were  averages  of  ratings  provided  by  supervisory,  self-,  and  peer  raters.  For 
example,  the  task  level  rating  form  for  the  Air  Traffic  Control  Operator  specialty  contained  15 
tasks  that  were  rated  by  supervisory,  self,  and  peer  sources.  The  15  ratings  were  averaged 
for  each  source  to  form  a  composite  rating.  The  three  composite  task  ratings  were  then  used 
to  predict  total,  hands-on,  and  interview  performance  in  separate  multiple  regression  analyses. 
The  squared  multiple  correlations  for  the  specialties  are  reported  in  Tables  B-1  to  B-8  of 
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Appendix  B.  These  correlations  reflect  the  extent  to  which  the  rating  form  composites  explain 
WTPT  performance. 

A  summary  of  squared  multiple  correlation  results  is  reported  in  Table  2.  Averages  of  the 
WTPT  correlations  were  computed  for  each  rating  form  and  each  specialty  by  transforming  the 
correlations  to  Fisher’s  Z  values,  computing  their  averages,  and  then  transforming  the  averages 
back  to  correlational  values.  As  shown  in  Table  2,  the  task  and  dimension  composites  were 
better  predictors  of  WTPT  performance  than  the  Air  Force  technical  and  global  composites. 
However,  none  of  the  rating  forms  was  adequate  in  predicting  WTPT  measures.  These  results 
suggest  that  rating  forms  should  not  be  considered  as  substitutes  or  surrogates  for  the  WTPT 
measures. 


Table  2.  Specialty  Averages  of  Squared  Multiple  Correlation  Coefficients  for 
Supervisor,  Self,  and  Peer  Composite  Predictors  of  WTPT  Measures  Obtained  with 
Task,  Dimension,  Air  Force-Wide  Technical  Knowledge, 
and  Global  Technical  Proficiency  Rating  Forms 


Specialty 

Rating 

Form 

Task 

Dimension 

Air  Force 

Global 

AIRTRAFF 

.083 

.082 

.066 

.056 

AVIONIC 

.127 

.143 

.141 

.084 

ISRADIO 

.088 

.147 

.149 

.140 

JETMECH 

.114 

.153 

.113 

.117 

AEROSPACE 

.174 

.174 

.156 

.183 

AIRCREW 

.089 

.069 

.019 

.043 

PERSON 

.147 

.120 

.073 

.125 

PRECISE 

.154 

.157 

.150 

.083 

Average 

.122 

.131 

.108 

.104 

Note  Averages  were  computed  from  Tables  8-1  through  B-8  for  total,  hands-on,  and 
interview  measures  obtained  with  walk-through  performance  testing. 


Rater  Source  Mean  Bias 


The  supervisory,  self-,  and  peer  ratings  for  each  rating  form  in  each  specialty  were  compared 
with  a  one-way,  repeated  measures  analysis  of  variance.  These  analyses  determined  whether 
the  rater  sources  differed  in  mean  bias  (i.e.,  relative  lenienrv  or  severity)  by  rating  form  and 
specialty.  For  example,  the  15  tasks  in  the  Air  Traffic  Control  Operator  specialty  were  each 
analyzed  to  determine  if  the  rater  sources  differed  in  mean  values. 

If  the  analysis  of  variance  F-test  indicated  that  the  source  means  differed  significantly 
(p  <  .05  or  p  <  .01),  post  hoc  comparisons  were  computed  among  the  means  using  Fisher  s 
least-significance-difference  (LSD)  approach.  Thus,  the  means  were  compared  with  t-tests 
using  a  level  for  significance  (i.e.,  p  <  .017  or  p  <  .003)  determined  by  the  number  of  possible 
comparisons  (i.e.,  3)  divided  into  the  stated  level  of  significance  tor  the  F-test.  Fisher's  LSD 
test  is  quite  conservative  (Winer,  1971;  p.  201),  and  it  probably  led  to  an  understatement  of 
the  number  of  significant  mean  differences.  However,  the  pattern  of  significant  differences  is 
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important  and  not  the  number  of  those  differences.  The  numbers  of  significant  differences  are 
reported  in  Tables  C-1  to  C-8  of  Appendix  C. 

The  general  pattern  of  the  significant  mean  differences  suggests  that  seif-ratings  tended  to 
be  greater  in  magnitude  than  supervisory  and  peer  ratings.  Further,  there  was  a  tendency 
for  peer  ratings  to  be  greater  than  supervisory  ratings. 


Multitrait-Multimethod  Validity  of  Rating  Forms 


The  construct  validity  of  each  rating  form  was  assessed  with  a  multitrait-multimethod  analysis 
(Dickinson,  1987).  The  traits  in  these  analyses  corresponded  to  the  constructs  appropriate  to 
the  rating  form.  For  example,  the  task  level  form  had  tasks  as  traits,  while  the  dimensional 
level  form  had  dimensions  as  traits.  The  methods  in  all  analyses  were  the  rating  sources 
(i.e.,  supervisory,  self,  and  peer  raters).  For  each  specialty,  the  analyses  utilized  the  measure 
intercorrelations  as  specified  by  the  trait  and  method  combinations.  These  analyses  are 

contained  in  Appendix  D  in  Tables  D-1  to  D-32. 

The  multitrait-multimethod  analyses  are  summarized  in  Tables  3,  4,  and  5.  These  tables 
report  the  extent  of  convergent  and  discriminant  validity  as  well  as  method  bias  of  each  rating 
form  for  each  specialty. 

As  shown  in  Table  3,  the  convergent  validity  was  moderate  (cf.  Dickinson,  Hassett,  & 
Tannenbaum,  1986).  This  validity  reflects  the  ability  of  the  measures  to  describe  individual 

differences  in  performance.  The  measures  were  equally  effective  for  the  specialties.  However, 

the  dimensional  level  rating  forms  appeared  to  possess  greater  convergent  validity  than  the 

remaining  rating  forms. 


Table  3.  Convergent  Validity  of  Rating  Forms  by  Specialty 


Rating  Form 


Specialty 

Task 

Dimension 

Air  Force 

Global 

Average 

AIRTRAFF 

.259 

.295 

.276 

.194 

.256 

AVIONIC 

.236 

.286 

.212 

.213 

.237 

ISRADIO 

.261 

.326 

.250 

.200 

.259 

JETMECH 

.195 

.350 

.280 

.265 

.272 

AEROSPACE 

.214 

.266 

.272 

.328 

.270 

AIRCREW 

.194 

.234 

.265 

.290 

.246 

PERSON 

.151 

.197 

.204 

.216 

.192 

PRECISE 

.212 

.256 

.225 

.250 

.236 

Average 

.215 

.276 

.248 

.244 

As  shown  in 

Table  4,  there  was  also  a  moderate  amount  of 

method  bias  (cf. 

Dickinson  et 

at.,  1986).  Of 

course,  method  bias  is  undesirable 

as  it  indicates  that  supervisory,  self,  and 

peer  raters  view 

'  incumbent  performance  differently. 

The  specialties 

did  not  differ  in  method 

bias.  However, 

the  Air  Force 

i  wide  rating  form  had 

less  method 

bias. 

Perhaps  this  was  due 

to  the  generic  nature  of  Air 

Force  wide  traits. 
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Table  4.  Method  Bias  of  Rating  Forms  by  Specialty 


Specialty 

— 

Rating  Form 

Average 

Task 

Dimension 

Air  Force 

Global 

AIRTRAFF 

.316 

.250 

.261 

.288 

.279 

AVIONIC 

.274 

.230 

.152 

.246 

.226 

ISRADIO 

.288 

.258 

.159 

.293 

.250 

JETMECH 

.280 

.230 

.231 

,260 

.250 

AEROSPACE 

.285 

.245 

.220 

.278 

.257 

AIRCREW 

.334 

.271 

.256 

.281 

.286 

PERSON 

.234 

.247 

.282 

.192 

.239 

PRECISE 

.254 

.258 

.232 

.227 

.243 

Average 

.283 

.249 

.224 

.258 

As  shown  in  Table  5,  there  was  little  evidence  for  discriminant  validity  (cf.  Dickinson  et 
al.,  1986).  The  traits  did  not  appear  to  describe  different  aspects  of  incumbent  performance. 
Further,  the  specialties  did  not  differ  in  discriminant  validity,  but  the  Air  Force-wide  and  global 
rating  forms  did  produce  somewhat  greater  discriminant  validity.  The  greater  validity  by  these 
rating  forms  was  probably  due  to  their  measurement  of  both  technical  and  interpersonal 
proficiency. 


Table  5.  Discriminant  Validity  of  Rating  Forms  by  Specialty 


Rating  Form 


Specialty 

Task 

Dimension 

Air  Force 

Global 

Average 

AIRTRAFF 

.057 

.067 

.090 

.085 

.075 

AVIONIC 

.092 

.014 

.165 

.136 

.102 

ISRADIO 

.032 

.015 

.173 

.090 

.078 

JETMECH 

.104 

.061 

.124 

.096 

.096 

AEROSPACE 

.073 

.071 

.083 

.062 

.072 

AIRCREW 

.077 

.075 

.089 

.027 

.067 

PERSON 

.093 

.047 

.099 

.165 

.101 

PRECISE 

.067 

.032 

.118 

.136 

.088 

Average 

.074 

.048 

.118 

.100 

In  sum,  the  rating  forms  possess  similar  multitrait-multimethod  properties  across  the  specialties. 
The  amount  of  convergent  validity  was  encouraging,  but  it  must  be  viewed  with  caution.  This 
validity  reflects  individual  differences  over  traits  and  methods.  Since  the  amount  of  method 
bias  was  much  greater  than  that  of  discriminant  validity,  convergent  validity  was  probably 
determined  more  by  the  different  viewpoints  of  the  rater  sources  than  by  traits  of  performance. 


Predictability  of  the  ASVAB 


The  focus  of  these  analyses  is  validating  the  ASVAB  for  the  eight  specialties.  The  analyses 
were  conducted  using  correlations  that  were  both  corrected  and  not  corrected  for  range 
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restriction.  As  noted  by  advisory  groups  to  the  Job  Performance  Measurement  project,  corrected 
correlations  provide  a  common  basis  for  interpretation  of  ASVAB  validities  for  specialties  that 
use  different  ASVAB  cutoff  scores  for  selection  and  classification.  The  Pearson-Lawley  procedure 
was  used  for  multivariate  correction  on  the  ten  subtests  of  the  ASVAB  (Mifflin  &  Verna,  1977). 
The  1980  youth  population  norms  were  used  to  perform  corrections  for  each  specialty. 

Validation  was  done  using  the  ten  ASVAB  subtests,  four  Air  Force  composites  (i.e.,  MAGE), 
and  AFQT  as  predictors  of  the  job  performance  measures.  Since  technical  training  school 
grades  have  traditionally  been  used  for  validating  ASVAB  predictors,  grades  were  also  used 
as  a  criterion  in  the  analysis. 

The  job  performance  measures  included  the  three  WTPT  measures  (i.e.,  total,  hands-on, 
and  interview)  and  the  job  knowledge  measure.  In  addition,  five  measures  were  constructed 
on  the  basis  of  the  factor  analyses.  Since  the  factor  analysis  for  the  Jet  Engine  Mechanic 
specialty  served  as  a  focus  for  interpretation  of  the  remaining  factor  analyses,  its  results  were 
used  to  define  the  five  criterion  composites  for  each  specialty.  The  measures  that  characterized 
each  factor  were  weighted  equally  to  form  the  composites. 


Correlations  Among  ASVAB  Predictors,  Job  Performance  Measures,  and  Training  School 
Grades 


The  uncorrected  correlations  between  the  ASVAB  predictors  and  job  performance  measures, 
factors,  and  training  school  grades  are  reported  in  Appendix  E,  while  the  corrected  correlations 
are  reported  in  Appendix  F.  As  shown  in  these  appendixes,  39%  of  the  correlations  between 
the  ASVAB  predictors  and  the  job  performance  measures  and  factors  were  statistically  significant, 
while  98%  of  the  correlations  between  the  ASVAB  piedictors  and  training  school  grades  were 
significant. 

The  pattern  of  these  significant  correlations  is  summarized  in  Table  6.  For  each  specialty, 
the  job  performance  measures,  factors,  and  training  school  grades  are  indicated  that  obtained 
at  least  one  statistically  significant  relationship  with  the  ASVAB  predictors. 


Table  6.  Pattern  of  Significant  Correlations  Among  ASVAB  Predictors,  Job  Performance 

Measures,  and  Training  School  Grades 


Job  Performance 

Specialty 

Measure  Factor 

TGRD 

AIRTRAFF 

NONE 

SUPER 

ALL 

AVIONIC 

TOT, HOA, INTA 

TECH 

ALL 

ISRADIO 

TOT, HOA, INTA 

TECH, SUPER, PEER 

ALL 

JETMECH 

TOT, INTA 

TECH, SELF 

ALL 

AEROSPACE 

TOT, HOA, INTA, JKT 

TECH, SELF, PEER 

ALL 

AIRCREW 

TOT, HOA, INTA, JKT 

TECH 

ALL 

PERSON 

TOT, HOA, INTA, JKT 

TECH, PEER 

ALL 

PRECISE 

TOT, HOA, INTA, JKT 

TECH, SELF, PEER, INP 

ALL 

Not*  Abbreviations  are:  TGRD,  Grade  received  in  tecnnicai  training;  NONE,  None  of  the  ASVAB 
predictors  correlated  significantly  with  the  job  performance  measures;  ALL,  All  ASVAB  predictors  correlated 
significantly  with  grades;  TOT,  Total;  HOA,  Hands-on;  INTA,  Interview;  and  JKT,  Job  knowledge  test. 
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The  correlations  between  the  ASVAB  predictors  and  the  job  performance  measures  were 
modest  in  magnitude  (i.e.,  ,10s  to  .30s).  The  Fisher  2  transformed  averages  of  the  uncorrected 
correlations  are  shown  in  Tables  7  and  8.  The  total,  hands-on,  and  interview  correlations  with 
the  ASVAB  predictors  were  similar  in  magnitude  across  the  specialties.  The  ASVAB  predictors 
tended  to  correlate  highest  with  the  job  knowledge  measures.  A  probable  explanation  for  this 
tendency  is  a  common  requirement  of  written  verbal  ability  for  ASVAB  predictors  and  the  job 
knowledge  measures. 

Of  the  job  performance  factors,  the  ASVAB  predictors  correlated  highest  with  the  technical 
proficiency  factor.  This  is  not  surprising,  since  this  factor  is  the  sum  of  hands-on  and  interview 
test  scores,  as  well  as  job  knowledge  scores  for  four  specialties.  The  ASVAB  predictors 
correlated  noticeably  lower  with  the  rating  source  (i.e.,  supervisory,  self,  and  peer)  and 
interpersonal  proficiency  factors. 


Table  7.  Average  Correlations  of  Job  Performance  Measures 
with  ASVAB  Predictors 


Speciality 

Total 

Hands-on 

Interview 

JKT 

AIRTRAFF 

.138 

.125 

.124 

_ 

AVIONIC 

.316 

.267 

.292 

- - 

ISRADIO 

.286 

.242 

.331 

— 

JETMECH 

.196 

.156 

.227 

— 

AEROSPACE 

.233 

.261 

.173 

.391 

AIRCREW 

.107 

.107 

.140 

.202 

PERSON 

.219 

.208 

.163 

.240 

PRECISE 

.326 

.317 

.269 

.364 

Average 

.230 

.212 

.216 

.302 

Table  8.  Average  Correlations  of  Job  Performance  Factors  and 
Training  School  Grades  With  ASVAB  Predictors 


Factors 


Speciality 

TECH 

SOURCE* 

INPERS 

TGRD 

AIRTRAFF 

.128 

.076 

.047 

.346 

AVIONIC 

.298 

.140 

.150 

.476 

ISRADIO 

.290 

.246 

.195 

.542 

JETMECH 

.220 

.110 

.099 

.513 

AEROSPACE 

.310 

.078 

.028 

.324 

AIRCREW 

.174 

.056 

.054 

.387 

PERSON 

.260 

.125 

.128 

.378 

PRECISE 

.356 

.231 

.115 

.408 

Average 

,262 

.136 

.103 

.393 

‘Average  of 

supervisory,  self,  and 

peer  factors. 
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The  ASVAB  predictors  correlated  much  higher  with  training  school  grades  (i.e.,  .30s  to  .60s) 
than  with  the  job  performance  measures  and  factors.  In  fact,  all  of  the  ASVAB  predictors 
correlated  significantly  with  grades  in  all  but  the  Information  Systems  Radio  Operator  (AFS 
492X1)  specialty.  In  that  specialty,  the  correlation  with  its  selection  composite  was  not  significant, 
probably  due  to  range  restriction  effects. 


Unique  Predictability  of  the  ASVAB  Subtests 


An  important  issue  is  whether  the  variance  explained  by  the  ASVAB  predictors  in  the  job 
performance  measures  and  training  school  grades  is  unique  to  the  type  of  criterion.  If  unique 
variance  is  accounted  for  by  the  ASVAB  predictors  in  each  type  of  criterion,  this  suggests  that 
the  criteria  are  not  interchangeable  for  selection  purposes.  This  is  an  important  issue,  since 
the  ASVAB  as  a  selection  tool  has  historically  been  validated  against  the  criterion  of  training 
school  grades. 

Multivariate  regression  analyses  were  conducted  to  assess  the  unique  predictability  of  the 
ASVAB  subtests.  The  analyses  were  conducted  with  uncorrected  and  corrected  correlations. 
In  these  analyses,  Roy-Bargman  step-down  tests  (Bock,  1975)  were  used  to  test  for  significant 
unique  predictability  of  the  ASVAB  subtests  for  the  WTPT  measures  and  training  school  grades. 
The  results  of  the  step-down  analyses  for  the  uncorrected  correlations  are  reported  in  Appendix 
G,  and  those  for  the  corrected  correlations  are  reported  in  Appendix  H. 

For  each  step-down  test,  squared  multiple,  partial  correlations  were  calculated.  These 
correlations  describe  the  unique  predictability  of  the  ten  ASVAB  subtests  for  the  WTPT  measures 
and  training  school  grades.  The  correlations  were  calculated  for  corrected  and  uncorrected 
ASVAB  subtest  scores.  Further,  the  hands-on  and  interview  partial  correlations  were  calculated 
only  with  training  school  grades  removed.  For  example,  the  results  immediately  after  the 
removal  of  grades  in  sets  4  of  the  step-down  analyses  were  used  to  calculate  the  partial 
correlations  for  the  hands-on  measures.  Only  removing  grades  emphasized  unique  predictability 
for  a  particular  WTPT  measure. 

Tables  9  and  10  indicate  that  the  subtests  uniquely  predicted  substantial  variance  in  training 
school  grades  after  removing  total  or  hands-on  and  interview  WTPT  scores.  That  is,  17%  to 
40%  of  unique  training  grade  variance  was  predicted  by  the  uncorrected  subtests,  and  43% 
to  79%  by  the  corrected  subtests.  However,  the  subtests  predicted  substantial  unique  WTPT 
variance  only  for  the  Avionic  Communications  specialty.  For  that  specialty,  25%  to  32%  of 
WTPT  score  variance  was  predicted  uniquely  by  the  uncorrected  subtests,  and  32%  to  46% 
by  the  corrected  subtests.  For  the  remaining  specialties,  ASVAB  subtests  accounted  for  much 
less  unique  WTPT  variance.  The  uncorrected  subtests  predicted  5%  to  21%  of  unique  WTPT 
variance,  and  the  corrected  subtests  predicted  4%  to  35%. 


IV.  RECOMMENDATIONS 

Based  on  the  findings,  recommendations  are  made  for  using  job  performance  measures 
and  training  school  grades  for  validating  the  ASVAB.  In  the  following  paragraphs,  the  structure 
of  the  job  performance  measures  is  summarized,  and  then,  recommendations  are  described. 
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Table  9.  Unique  Predictive  Effectiveness  of  10  ASVAB  Subtests 
for  WTPT  Measures  and  Training  School  Grades 
(Uncorrected  for  Restriction) 


Speciality 

TOTWTPT 

WTPT  Measures 

Hands-on 

Interview 

TGRD1* 

TGRD2* 

AIRTRAFF 

.099 

.113* 

.093 

.206** 

.202** 

AVIONIC 

.319** 

.250* 

.276** 

.393** 

.401** 

ISRADIO 

.105 

.101 

.149 

.377** 

.365** 

JETMECH 

.079 

.061 

.053 

.333** 

.336** 

AEROSPACE 

.129** 

.143** 

.088* 

.177** 

.170** 

AIRCREW 

.052 

.052 

.112* 

.198** 

.197** 

PERSON 

.090 

.056 

.097 

.254** 

.253** 

PRECISE 

.206** 

.201** 

.124 

.343** 

.334** 

“Correlations  for  training  school  grades  obtained  from  Roy-Bargman  analysis  sets  1  and  2. 
*p  <  .05. 

**p  <  .01. 


Table  10.  Unique  Predictive  Effectiveness  of  10  ASVAB  Subtests 
for  WTPT  Measures  and  Training  School  Grades 
(Corrected  for  Restriction) 


WTPT  Measures 


Speciality 

TOTWTPT 

Hands-on 

Interview 

TGRDla 

TGRD2 

AIRTRAFF 

.098 

.123* 

.086 

.575** 

.556** 

AVIONIC 

.435** 

.325* 

.463** 

.772** 

.786** 

ISRADIO 

.216 

.209 

.278 

.658** 

.626** 

JETMECH 

.137 

.115 

.097 

.641** 

.639** 

AEROSPACE 

.187** 

.220** 

.122* 

.453** 

.429** 

AIRCREW 

.039 

.039 

.098* 

.507** 

.507** 

PERSON 

.187 

.074 

.254 

.442** 

.448** 

PRECISE 

.347** 

.341** 

.212 

.620** 

.603** 

“Correlations  for 

training  school 

grades  obtained  from 

Roy-Bargman  analysis 

sets  1  and  2. 

*p  <  .05. 
**p  <  .01. 


For  all  specialties,  the  factor  analyses  showed  that  a  structure  of  four  distinct  factors  was 
assessed  by  the  job  performance  measures.  Importantly,  a  technical  proficiency  factor  was 
found  that  was  defined  by  the  hands-on  and  interview  components  of  the  WTPT  and  by  the 
job  knowledge  test.  The  hands-on  component  tended  to  have  the  greater  loadings  on  this 
proficiency  factor.  This  is  encouraging,  since  the  hands-on  component  is  argued  to  be  the 
high  fidelity  measure  of  technical  proficiency  (Wigdor  &  Green,  1986). 

The  remaining  three  factors  reflected  supervisory,  self,  and  peer  points  of  view.  These 
rater  source  factors  were  distinct  from  the  technical  proficiency  factor  and  correlated  both 
negatively  and  positively  with  that  factor.  Thus,  the  rater  source  factors  did  not  consistently 
reflect  a  unique  perspective  of  technical  proficiency. 
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Although  the  job  performance  measures  were  designed  to  measure  interpersonal  proficiency, 
a  distinct  interpersonal  factor  appeared  only  for  the  Jet  Engine  Mechanic  specialty.  The 
remaining  specialties  had  a  hybrid  interpersonal  factor  that  reflected  a  single  perspective  (i.e., 
supervisory,  self,  or  peer).  This  suggests  that  the  rating  forms  were  not  effective  measures 
of  interpersonal  proficiency. 

Analyses  focusing  solely  on  the  rating  measures  revealed  a  consistent  leniency  bias. 
Incumbents  rated  themselves  higher  than  peers  and  supervisors,  while  peers  rated  the  incumbents 
higher  than  supervisors. 

The  rating  measures  also  demonstrated  very  little  discriminant  validity,  indicating  that  no 
matter  the  level  used  for  describing  performance  (i.e.,  task,  dimension,  global,  or  Air  Force-wide) 
distinct  aspects  of  performance  could  not  be  described  with  these  measures.  Although  moderate 
convergent  validity  was  obtained  in  ordering  the  incumbents  at  all  levels  of  performance,  this 
ordering  was  likely  due  to  the  biased  perspectives  of  the  rater  sources. 

With  respect  to  valida...ig  the  ASVAB,  the  technical  proficiency  factor,  as  well  as  its  defining 
measures  was  correlated  with  the  ASVAB  for  seven  of  the  eight  specialties.  Although  these 
validity  correlations  were  modest  in  magnitude,  they  suggest  that  the  ASVAB  does  predict 
hands-on  job  performance. 

In  contrast,  the  rater  source  factors  showed  very  small  correlations  with  the  ASVAB.  The 
peer  factor  was  predicted  by  the  ASVAB  for  four  specialties,  the  self  factor  for  three,  and  the 
supervisory  factor  for  two  specialties.  This  inconsistent  pattern  of  validity  correlations  across 
the  specialties  indicates  that  rating  measures  should  not  be  recommended  for  validating  the 
ASVAB. 

In  sum,  only  three  of  the  job  performance  measures  appear  to  be  acceptable  for  validating 
the  ASVAB.  These  measures  reflect  hands-on  performance  or  procedural  knowledge  (i.e., 
interview  and  job  knowledge  tests)  of  that  performance,  rather  than  recalled  judgments  of  that 
performance  (i.e.,  ratings). 

Finally,  the  training  outcome  of  technical  school  grades  was  predicted  best  by  the  ASVAB 
measures  for  all  specialties.  Even  when  technical  proficiency  was  partialled  from  the  correlations, 
the  ASVAB  still  predicted  training  school  grades  for  all  specialties.  In  contrast,  when  training 
school  grades  were  removed  from  the  job  performance  measures,  the  ASVAB  predicted  only 
three  of  the  eight  specialties.  Clearly,  the  ASVAB  is  most  useful  for  predicting  success  in 
training  schools. 
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Table  A-1.  Air  Traffic  Control  Operators  (AFS  272X0):  Factor  Structure 
of  the  Job  Performance  Measures 


Factors 


Variables 

TECH 

SUPER 

SELF 

PEER 

PEER-TEC 

Hands-on 

.804 

.194 

.228 

-.107 

-.135 

Interview 

.995 

.144 

.229 

-.035 

-.124 

T-self 

,278 

.242 

.845 

-.105 

-.368 

T-supervisor 

.263 

.872 

.358 

-.279 

-.368 

T-peer 

.345 

.495 

.484 

-.517 

-.815 

D-self 

.284 

.263 

.878 

-.060 

-.361 

D-supervisor 

.256 

.929 

.336 

-.333 

-.332 

D-peer 

.289 

.564 

.444 

-.510 

-.768 

A-self-TK 

.205 

.296 

.745 

-.158 

-.302 

A-self-IP 

.117 

.127 

.756 

-.132 

.051 

A-supervisor-TK 

.282 

.825 

.325 

-.290 

-.380 

A-supervisor-IP 

.063 

.730 

.161 

-.429 

.042 

A-peer-TK 

.162 

.427 

.305 

-.702 

-.492 

A-peer-IP 

.102 

.305 

.085 

-.853 

-.050 

G-self-TK 

.226 

.263 

.752 

-.153 

-.351 

G-self-IP 

.232 

.217 

.572 

-.164 

-.084 

G-supervisor-TK 

.201 

.805 

.323 

-.400 

-.364 

G-supervisor-IP 

.069 

.611 

.071 

-.326 

.070 

G-peer-TK 

.219 

.463 

.350 

-.691 

-.397 

G-peer-IP 

.003 

.325 

.083 

-.764 

-.060 

Note.  N  =  176.  Underlined  loadings  indicate  variables  that  characterize  a  factor.  Abbreviations  for  factors  are: 
TECH,  proficiency  of  the  technical  aspects  of  work;  SUPER,  overall  performance  from  supervisory  point  of  view;  SELF, 
overall  performance  from  incumbent  point  of  view;  PEER,  overall  performance  from  a  peer  point  of  view;  and  PEER-TEC, 
proficiency  on  technical  aspects  of  work  from  peer  point  of  view.  Abbreviations  for  variables  are;  T,  task-level  ratings; 
D,  task-oriented  dimensional-level  ratings;  A,  Air  Force-wide  dimensional-level  ratings;  Q,  global-level  ratings;  TK,  technical 
knowledge  and  skill;  and  IP,  interpersonal  aspects  of  work. 


Table  A-2.  Air  Traffic  Control  Operators  (AFS  272X0):  Correlations  Among  the 
Factors  of  the  Job  Performance  Measures 


Factors 

TECH 

SUPER 

SELF 

PEER 

PEER-TEC 

TECH 

1.000 

SUPER 

.208 

1.000 

SELF 

.275 

.292 

1.000 

PEER 

-.092 

-.420 

-.173 

1.000 

PEER-TEC 

-.191 

-.240 

-.282 

.148 

1.000 

Note.  Abbreviations  for  factors  are:  TECH,  proficiency  of  the  technical  aspects  of  work;  SUPER,  overall  performance 
from  supervisory  point  of  view;  SELF,  overall  performance  from  incumbent's  point  of  view,  DCCP  overall  performance 
from  a  peer  point  of  view;  and  PEER-TEC,  proficiency  on  technical  aspects  of  work  from  peer  point  of  view. 


Table  A-3.  Avionic  Communications  Specialists  (AFS  328X0):  Factor  Structure  of 

the  Job  Performance  Measures 


Factors 


Variables 

TECH 

SUPER 

SELF 

PEER-TEC 

PEER-INP 

Hands-on 

.984 

.233 

.109 

.262 

.022 

Interview 

.636 

.188 

.141 

.257 

.110 

T-self 

.329 

.303 

.771 

.301 

.154 

T-supervisor 

.288 

.904 

.243 

.484 

-.077 

T-peer 

.318 

.441 

.219 

.940 

-.274 

D-self 

.348 

.332 

.772 

.372 

.201 

D-supervisor 

.349 

.863 

.339 

.537 

.007 

D-peer 

.377 

.519 

.279 

.941 

-.192 

A-self-TK 

.415 

.398 

.789 

.324 

.127 

A-self-IP 

-.012 

.134 

.683 

.023 

-.116 

A-supervisor-TK 

.373 

.862 

.334 

.433 

.046 

A-supervisor-IP 

.096 

.693 

.235 

.208 

-.433 

A-peer-TK 

.220 

.456 

.404 

.695 

-.369 

A-peer-IP 

.036 

.305 

.213 

.510 

-.844 

G-self-TK 

.374 

.416 

.599 

.418 

.143 

G-self-IP 

-.100 

.098 

.525 

.103 

-.132 

G-supervisor-TK 

.303 

.752 

.340 

.460 

.137 

G-supervisor-IP 

-.005 

.528 

.094 

.236 

-.484 

G-peer-TK 

.200 

.458 

.197 

.678 

-.457 

G-peer-IP 

-.059 

.107 

-.030 

.335 

-.791 

Note.  N  =  86.  Underlined  loadings  indicate  variables  that  characterize  a  factor.  Abbreviations  for  factors  are: 
TECH,  proficiency  of  the  technical  aspects  of  work;  SUPER,  overall  performance  from  supervisory  point  of  view;  SELF, 
overall  performance  from  incumbent's  point  of  view;  PEER-TEC,  technical  performance  from  a  peer  point  of  view;  and 
PEER-INP,  proficiency  on  interpersonal  aspects  of  work  from  peer  point  of  view.  Abbreviations  for  variables  are:  T, 
task-level  ratings;  D,  task-oriented  dimensional-level  ratings;  A,  Air  Force-wide  dimensional-level  ratings;  G,  alobal-level 
ratings;  TK,  technical  knowledge  and  skill;  and  IP,  interpersonal  aspects  of  work. 


Table  A-4.  Avionic  Communications  Specialists  (AFS  328X0):  Correlations  Among  the 

Factors  of  the  Job  Performance  Measures 


Factors 

TECH 

SUPER 

SELF 

PEER-TEC 

PEER-INP 

TECH 

1.000 

SUPER 

.268 

1.000 

SELF 

.169 

.299 

1.000 

PEER-TEC 

.328 

.431 

.256 

1.000 

PEER-INP 

.160 

-.134 

.004 

-.174 

1.000 

Note.  Abbreviations  for  factors  are:  TECH,  proficiency  of  the  technical  aspects  of  work;  SUPER,  overall  performance 
from  supervisory  point  of  view;  SELF,  overall  performance  from  incumbent's  point  of  view;  PEER-TEC,  technical 
performance  from  a  peer  point  of  view;  and  PEER-INP,  proficiency  on  interpersonal  aspects  of  work  from  peer  point 
of  view. 
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Table  A-5.  Information  Systems  Radio  Operators  (AFS  492X1):  Factor  Structure 

of  the  Job  Performance  Measures 


Factors 


Variables 

TECH 

SUP-TEC 

SELF 

PEER 

SUP-INP 

Hands-on 

.807 

.279 

.168 

.288 

-.074 

Interview 

.992 

.343 

.062 

.297 

-.027 

T-self 

.316 

.456 

.765 

.278 

-.335 

T-supervisor 

.388 

.862 

.337 

.379 

.038 

T-peer 

.323 

.547 

.362 

.792 

-.160 

D-self 

.314 

.451 

.825 

.244 

-.358 

D-supervisor 

.353 

.872 

.295 

.424 

.087 

D-peer 

.464 

.600 

.417 

.816 

-.210 

A-self-TK 

.275 

.359 

.722 

.378 

-.253 

A-self-IP 

.058 

.154 

.740 

.136 

.207 

A-supervisor-TK 

.376 

.874 

.273 

.390 

.068 

A-supervisor-IP 

.072 

.483 

.222 

.292 

.706 

A-peer-TK 

.419 

.419 

.230 

.754 

-.054 

A-peer-IP 

.192 

.135 

.114 

.686 

.295 

G-self-TK 

.379 

.495 

.626 

.365 

-.318 

G-self-IP 

-.045 

.037 

.551 

.208 

.171 

G-supervisor-TK 

.372 

.874 

.233 

.354 

.025 

G-supervisor-IP 

.174 

.345 

.077 

.364 

.539 

G-peer-TK 

.374 

.523 

.257 

.708 

-.097 

G-peer-IP 

.137 

.113 

.189 

.562 

.234 

Note.  N  =  139.  Underlined  loadings  indicate  variables  that  characterize  a  factor.  Abbreviations  for  factors  are: 
TECH,  proficiency  of  the  technical  aspects  of  work;  SUP-TEC,  technical  proficiency  from  supervisory  point  of  view; 
SELF,  overall  performance  from  incumbent's  point  of  view;  PEER,  overall  performance  from  a  peer  point  of  view;  and 
SUP-INP,  proficiency  on  interpersonal  aspects  of  work  from  supervisory  point  of  view.  Abbreviations  for  variables  are: 
T,  task-level  ratings;  0,  task-oriented  dimensional-level  ratings;  A,  Air  Force-wide  dimensional-level  ratings;  G,  global 
level  ratings;  TK,  technical  knowledge  and  skill;  and  IP,  interpersonal  aspects  of  work. 


Table  A-6.  information  Systems  Radio  Operators  (AFS  492X1):  Correlations  Among  the 

Factors  of  the  Job  Performance  Measures 


Factors 

TECH 

SUP-TEC 

SELF 

PEER 

SUP-INP 

TECH 

1.000 

SUP-TEC 

.383 

1.000 

SELF 

.151 

.294 

1.000 

PEER 

.331 

.381 

.281 

1.000 

SUP-INP 

-.115 

-.027 

-.049 

.102 

1.000 

Note.  Abbreviations  for  factors  are:  TECH,  proficiency  of  the  technical  aspects  of  work;  SUP-TEC,  technical 
proficiency  from  supervisory  point  of  view;  SELF,  overall  performance  from  incumbents  point  of  view,  PEER,  overall 
performance  from  a  peer  point  of  view;  and  SUP-INP,  proficiency  on  interpersonal  aspects  of  work  from  supervisory 
point  of  view. 
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Table  A-7.  Jet  Engine  Mechanics  (AFS  426X2):  Factor  Structure  of 
the  Job  Performance  Measures 


Factors 


Variables 

TECH 

SUPER 

SELF 

PEER 

INPERS 

Hands-on 

.556 

.308 

.173 

.312 

-.077 

Interview 

.986 

.180 

.146 

.192 

-.092 

T-self 

.275 

.305 

.860 

.288 

-.095 

T-supervisor 

.334 

.836 

.395 

.368 

-.096 

T-peer 

.346 

.396 

.357 

.828 

-.033 

D-self 

.243 

.340 

.873 

.320 

-.100 

□-supervisor 

.301 

.888 

.452 

.482 

-.102 

D-peer 

.397 

.449 

.400 

.884 

-.095 

A-self-TK 

.252 

.406 

.803 

.368 

-.029 

A-self-IP 

-.134 

.278 

.593 

.169 

.526 

A-supervisor-TK 

.279 

.834 

.400 

.442 

.039 

A-supervisor-IP 

.072 

.815 

.152 

.428 

.431 

A-peer-TK 

.261 

.476 

.284 

.783 

-.028 

A-peer-IP 

.078 

.306 

.097 

.648 

.495 

G-self-TK 

.169 

.292 

.751 

.242 

-.063 

G-self-IP 

-.023 

.236 

.488 

.198 

.370 

G-supervisor-TK 

.290 

.810 

.328 

.482 

-.046 

G-supervisor-IP 

.082 

.675 

.069 

.373 

.443 

G-peer-TK 

.278 

.501 

.343 

.724 

-.062 

G-peer-lP 

.015 

.308 

.041 

.576 

.348 

Not*.  N  =  216.  Underlined  loadings  indicate  variables  that  characterize  a  factor.  Abbreviations  for  factors  are: 
TECH,  proficiency  of  the  technical  aspects  of  work;  SUPER,  overaJI  performance  from  supervisory  point  of  view;  SELF, 
overall  performance  from  incumbent's  point  of  view;  PEER,  overall  performance  from  a  peer  point  of  view;  and  INPEPS, 
overaJI  proficiency  on  interpersonal  aspects  of  work  from  supervisory,  incumbent,  and  peer  points  of  view.  Abbreviations 
for  vanables  are:  T,  task-leval  ratings;  D,  task-oriented  dimensional-level  ratings;  A,  Air  Force-wide  dimensional-level 
ratings;  G,  global  level  ratings;  TK,  technical  knowledge  and  skill;  and  IP,  interpersonal  aspects  of  work. 


Table  A-8.  Jet  Engine  Mechanics  (AFS  426X2):  Correlations  Among  the  Factors  of 

the  Job  Performance  Measures 


Variables 

TECH 

SUPER 

SELF 

PEER 

INPERS 

TECH 

1.000 

SUPER 

.242 

1.000 

SELF 

.175 

.329 

1.000 

PEER 

.266 

.479 

.270 

1.000 

INPERS 

-.211 

.118 

-.014 

.115 

1.000 

Note.  Abbreviations  for  factors  are:  TECH,  proficiency  of  the  technical  aspects  of  work;  SUPER,  overall  performance 
from  supervisory  point  of  view;  SELF,  overall  performance  from  incumbent's  point  of  view;  PEER,  overall  performance 
from  a  peer  point  of  view;  and  INPERS,  overall  proficiency  on  interpersonal  aspects  of  work  from  supeivisory,  incumbent, 
and  peer  points  of  vie.-.. 
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Table  A-9.  Aerospace  Ground  Equipment  Mechanics  (AFS  423X5):  Factor  Structure  of 

the  Job  Performance  Measures 


Factors 


Variables 

TECH 

SUPER 

SELF 

PEER 

PEER-TEC 

Hands-on 

-.820 

.274 

.284 

.168 

.234 

Interview 

-.836 

.258 

.338 

.181 

.254 

JKT 

-.510 

.181 

.216 

.102 

.303 

T-self 

-.368 

.242 

.808 

.100 

.318 

T-supervisor 

-.337 

.893 

.366 

.309 

.253 

T-peer 

-.420 

.490 

.422 

.556 

.806 

D-self 

-.397 

.344 

.858 

.155 

.362 

D-supervisor 

-.368 

.945 

.370 

.329 

.292 

D-peer 

-.428 

.509 

.463 

.589 

.827 

A-self-TK 

-.371 

.371 

.693 

.175 

.308 

A-self-IP 

-.104 

.402 

.654 

.414 

-.015 

A-supervisor-TK 

-.305 

.872 

.374 

.374 

.263 

A-supervisor-IP 

-.145 

.808 

.337 

.481 

.022 

A-peer-TK 

-.398 

.438 

.390 

.612 

.540 

A-peer-IP 

-.136 

.400 

.215 

.878 

.206 

G-self-TK 

-.381 

.342 

.687 

.245 

.184 

G-self-IP 

-.122 

.258 

.603 

.224 

-.021 

G-supervisor-TK 

-.281 

.828 

.345 

.364 

.247 

G-supervisor-IP 

-.079 

.784 

.348 

.445 

.033 

G-peer-TK 

-.403 

.413 

.294 

.637 

.523 

G-peer-lP 

-.229 

.381 

.271 

.758 

.240 

Note  N  =  247  Underlined  loadings  indicate  variables  that  characterize  a  factor.  Abbreviations  for  factors  are: 
TECH,  proficiency  of  the  technical  aspects  of  work;  SUPER,  overall  performance  from  supervisory  point  of  view;  SELF, 
overall  performance  from  incumbent  point  of  view;  PEER,  overall  performance  from  a  peer  point  of  view;  and  PEER-TEC, 
proficiency  on  technical  aspects  of  work  from  peer  point  of  view.  Abbreviations  for  variables  are:  JKT,  job  knowledge 
test;  T,  task-level  ratings;  □,  task-oriented  dimensional-level  ratings;  A,  Air  Force-wide  dimensional-level  ratings;  G, 
global-level  ratings;  Th,  technical  knowledge  and  skill;  and  IP,  interpersonal  aspects  of  work. 


Table  A- 10.  Aerospace  Ground  Equipment  Mechanics  (AFS  423X5): 
Correlations  Among  the  Factors  the  Job  Performance  Measures 


Variables 

TECH 

SUPER 

SELF 

PEER 

PEER-TEC 

TECH 

1.000 

SUPER 

-.284 

1.000 

SELF 

-.341 

.405 

1.000 

PEEP 

-.158 

.442 

.277 

1.000 

PEER-TE 

-.389 

.212 

.208 

.209 

1.000 

Note  Abbreviations  for  factors  are:  TECH,  proficiency  of  the  technical  aspects  of  work;  SUPER,  overall  performance 
from  supervisory  point  of  view;  SELF,  overall  performance  from  incumbent's  point  of  view;  PEER,  overall  performance 
from  a  peer  point  of  view,  and  PEER-TEC,  proficiency  on  technical  aspects  of  work  from  peer  point  of  view. 
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Table  A-11.  Aircrew  Life  Support  Specialists  (AFS  122X0):  Factor  Structure  of 

the  Job  Performance  Measures 


Factors 


Variables 

TECH 

SUPER 

SELF 

PEER 

PEER-TEC 

Hands-on 

.619 

.113 

.116 

-.053 

-.184 

Interview 

.929 

.061 

.220 

-.065 

-.188 

JKT 

.723 

.148 

.110 

.025 

-.400 

T-self 

.275 

.159 

.821 

-.020 

-.178 

T-supervisor 

.163 

.838 

.285 

-.149 

-.226 

T-peer 

.344 

.394 

.306 

-.449 

-.861 

D-self 

.250 

.204 

.756 

-.039 

-.276 

D-supervisor 

.188 

.849 

.381 

-.240 

-.412 

D-peer 

.311 

.414 

.374 

-.536 

-.824 

A-self-TK 

.094 

.298 

.581 

-.260 

-.140 

A-self-IP 

-.075 

.342 

.541 

-.341 

-.073 

A-supervisor-TK 

.081 

.810 

.285 

-.274 

-.252 

A-supervisor-IP 

.024 

.795 

.240 

-.414 

-.097 

A-peer-TK 

.076 

.302 

.251 

-.794 

-.277 

A-peer-IP 

-.003 

.360 

.235 

-.893 

-.183 

G-self-TK 

.086 

.265 

.572 

-.277 

-.109 

G-self  IP 

.014 

.163 

.391 

-.265 

.046 

G-sjpervisor-TK 

.129 

.824 

.284 

-.258 

-.212 

G-supervisor-IP 

.022 

.751 

.223 

-.294 

-.053 

G-peer-TK 

.147 

.288 

.290 

-.744 

-.364 

G-peer-IP 

.010 

.288 

.263 

-.819 

-.099 

Note.  N*  179.  Underlined  loadings  indicate  variables  that  characterize  a  factor.  Abbreviations  for  factors  are: 
TECH,  proficiency  of  the  technical  aspects  of  work;  SUPER,  overall  performance  from  supervisory  point  of  view;  SELF, 
overall  performance  from  incumbent  point  of  view;  PEER,  overall  performance  from  a  peer  point  of  view;  and  PEER-TEC. 
proficiency  on  technical  aspects  of  work  from  peer  point  of  view.  Abbreviations  for  variables  are:  JKT,  job  knowledge 
test;  T,  task-level  ratings;  D,  task-oriented  dimensional-level  ratings;  A,  Air  Force-wide  dimensional -level  ratings;  G. 
global-level  ratings;  TK,  technical  knowledge  and  skill;  and  IP,  interpersonal  aspects  of  work. 


Table  A- 12.  Aircrew  Life  Support  Specialists  (AFS  122X0):  Correlations  Among  the 
Factors  of  the  Job  Performance  Measures 


Variables 

TECH 

SUPER 

SELF 

PEER 

PEER-TEC 

TECH 

1.000 

SUPER 

.099 

1.000 

SELF 

.166 

.330 

1.000 

PEER 

-.001 

-.337 

-.277 

1.000 

PEER-TEC 

-.315 

-.227 

-.168 

.167 

1.000 

Note.  Abbreviations  for  factors  are:  TECH,  proficiency  of  the  technical  aspects  of  work;  SUPER,  overall  performance 
from  supervisory  point  of  view;  SELF,  overall  performance  from  incumbent's  point  of  view;  PEER,  overall  performance 
from  a  peer  point  of  view;  and  PEER-TEC,  proficiency  on  technical  aspects  of  work  from  peer  point  of  view. 
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Table  A- 13.  Personnel  Specialists  (AFS  7234X0):  Factor  Structure  of 
the  Job  Performance  Measures 


Factors 


Variables 

TECH 

SUPER 

SELF 

PEER-TEC 

PEER-INP 

Hands-on 

.998 

.114 

.167 

-.291 

-.117 

Interview 

.283 

.247 

.133 

-.181 

-.231 

JKT 

.512 

.057 

.042 

-.096 

-.014 

T-self 

.284 

.036 

.796 

-.275 

-.178 

T-supervisor 

.202 

.806 

.125 

-.446 

-.111 

T-peer 

.388 

.317 

.302 

-.657 

.028 

D-self 

.226 

.047’ 

.841 

-.337 

-.279 

D-supervisor 

.154 

.910 

.044 

-.431 

-.089 

D-peer 

.216 

.460 

.298 

-.803 

.065 

A-self-TK 

.141 

.089 

.662 

-.443 

-.129 

A-self-IP 

-.030 

.109 

.793 

-.126 

.216 

A-supervisor-TK 

.143 

.824 

.070 

-.466 

.084 

A-supervisor-IP 

.110 

.792 

.055 

-.144 

.330 

A-peer-TK 

.181 

.331 

.234 

-.793 

.229 

A-peer-IP 

.102 

.355 

.130 

-.466 

.693 

G-self-TK 

.225 

.013 

.562 

-.322 

-.112 

G-self-IP 

-.006 

.107 

.506 

-.046 

.240 

G-supervisor-TK 

.124 

.788 

.150 

-.324 

.119 

G-supervisor-IP 

.008 

.674 

.013 

-.089 

.360 

G  peer-TK 

.234 

.298 

.264 

-.723 

.141 

G-peer-lP 

.035 

.261 

.110 

-.353 

.632 

Not*.  N  =  185.  Underlined  loadings  indicate  variables  that  characterize  a  factor.  Abbreviations  for  factors  are: 
TECH,  proficiency  of  the  technical  aspects  of  work;  SUPER,  overall  performance  from  supervisory  point  of  view;  SELF, 
overall  performance  from  incumbent  point  of  view;  PEER-TEC,  proficiency  on  technical  aspects  of  work  from  peer  point 
of  view;  and  PEER-INP,  proficiency  on  interpersonal  aspects  of  work  from  peer  point  of  view.  Abbreviations  for  variables 
are:  JKT,  job  knowledge  test;  T,  task-level  ratings;  D,  task-oriented  dimensional-level  ratings;  A,  Air  Force-wide 
dimensional-level  ratings;  G,  global-level  ratings;  TK,  technical  knowledge  and  skill;  and  IP,  interpersonal  aspects  of 
work. 


Table  A- 14.  Personnel  Specialists  (AFS  7234X0):  Correlations  Among  the 
Factors  of  the  Job  Performance  Measures 


Variables 

TECH 

SUPER 

SELF 

PEER-TEC 

PEER-INP 

TECH 

1.000 

SUPER 

.147 

1.000 

SELF 

.155 

.090 

1.000 

PEER-TEC 

-.273 

-.338 

-.285 

1.000 

PEER-INP 

-.130 

.144 

-.0G3 

-.027 

1.000 

Note.  Abbreviations  for  factors  are:  TECH,  proficiency  of  the  technical  aspects  of  work;  SUPER,  overall  performance 
from  supervisory  point  of  view;  SELF,  overall  performance  from  incumbent's  point  of  view;  PEER-TEC,  technical 
performance  from  a  peer  point  of  view;  and  PEER-INP,  proficiency  on  interpersonal  aspects  of  work  from  peer  point 
of  view. 
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Table  A- 15.  Precision  Measurement  Equipment  Laboratory  Specialists  (AFS  324X0): 
Factor  Structure  of  the  Job  Performance  Measures 


Factors 


Variables 

TECH 

SUPER 

SELF 

PEER-TEC 

PEER-INP 

Hands-on 

,712 

.248 

.107 

.287 

-.202 

Interview 

.673 

.193 

-.060 

.254 

.027 

JKT 

.739 

.325 

.079 

.340 

-.201 

T-self 

,345 

.242 

.705 

.312 

-.451 

T-supervisor 

.342 

.784 

.309 

.346 

-.097 

T-peer 

.464 

.290 

.205 

.846 

-.040 

D-self 

.344 

.279 

.704 

.336 

-.580 

D-supervisor 

.379 

.872 

.301 

.386 

-.133 

D-peer 

.450 

.360 

.194 

.804 

-.138 

A-self-TK 

.275 

.315 

.452 

.292 

-.679 

A-self-IP 

-.032 

.409 

.747 

.152 

-.008 

A-supervisor-TK 

.451 

.754 

.285 

.342 

-.128 

A-supervisor-IP 

-.006 

.708 

.407 

.228 

.425 

A-peer-TK 

.348 

.255 

.202 

.881 

-.220 

A-peer-IP 

.133 

.343 

.170 

.637 

.397 

G-self-TK 

.259 

.435 

.513 

.314 

-.585 

G-self-IP 

-.095 

.186 

.574 

.094 

.004 

G-supervisor-TK 

.254 

.812 

.271 

.286 

-.056 

G-supervisor-IP- 

.066 

.580 

.429 

.159 

.328 

G-peer-TK 

.301 

.314 

.202 

.826 

-.283 

G-peer-IP 

.143 

.229 

.192 

.608 

.320 

Note.  N  *  135.  Underlined  loadings  indicate  variables  that  characterize  a  factor.  Abbreviations  for  factors  are: 
TECH,  proficiency  of  the  technical  aspects  of  work;  SUPER,  overall  performance  from  supervisory  point  of  view;  SELF, 
overall  performance  from  incumbent  point  of  view;  PEER,  overall  performance  from  a  peer  point  of  view;  and  SELF-TEC, 
proficiency  on  technical  aspects  of  work  from  incumbent  point  of  view  as  contrasted  from  supervisory  and  peer  points 
of  view.  Abbreviations  for  variables  are:  JKT,  job  knowledge  test;  T,  task  level  ratings;  D,  task-oriented  dimensional-level 
ratings;  A,  Air  Force-wide  dimensional-level  ratings;  G,  global-level  ratings;  TK,  technical  knowledge  and  skill;  and  IP, 
interpersonal  aspects  of  work. 


Table  A- 16.  Precision  Measurement  Equipment  Laboratory  Specialists  (AFS  324X0): 
Correlations  Among  the  Factors  of  the  Job  Performance  Measures 


Variables 

TECH 

SUPER 

SELF 

PEER-TEC 

PEER-INP 

TECH 

1.000 

SUPER 

.263 

1.000 

SELF 

.026 

.385 

1.000 

PEER 

.361 

.335 

.237 

1.000 

SELF-TEC 

-.232 

.022 

-.127 

-.045 

1.000 

from  supervisory  point  of  view;  SELF,  overall  performance  from  incumbent’s  point  of  view;  PEER,  overall  performance 
from  a  peer  point  of  view;  and  SELF-TEC,  proficiency  on  technical  aspects  of  work  from  incumbent  point  of  view  as 
contrasted  from  supervisory  and  peer  points  of  view. 
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APPENDIX  B:  SQUARED  MULTIPLE  CORRELATIONS  FOR 
WTPT  MEASURES  USING  COMPOSITE  RATINGS 
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Table  B-1.  Air  Traffic  Control  Operators  (AFS  272X0):  Squared  Multiple 
Correlation  Coefficients  for  Walk-Through  Performance  Test  Measures 
Using  the  Composite  Ratings  of  Self,  Peer,  and  Supervisor  Sources 
as  Predictors  Obtained  with  Task,  Dimension,  Air  Force-Wide 
Technical  Knowledge,  and  Global  Technical  Proficiency  Rating  Forms 


WTPT 

Measures 

Rating  Form 

Task 

Dimension 

Air  Force 

Global 

TOTWTPT 

.086** 

.088** 

.071** 

.056** 

Hands-on 

.080** 

.088** 

.070** 

.065** 

Interview 

.083** 

.070** 

.057* 

.046* 

Note.  N 

ranged  from  183  to 

184.  Abbreviation: 

TOTWTPT,  Total 

score  obtained 

with  Walk-Through  Performance  Testing. 
*p  <  .05. 

**p  <  .01. 


Table  B-2.  Avionic  Communications  Specialists  (AFS  328X0):  Squared 
Multiple  Correlation  Coefficients  for  Walk-Through  Performance  Test 
Measures  Using  the  Composite  Ratings  of  Self,  Peer,  and  Supervisor 
Sources  as  Predictors  Obtained  with  Task,  Dimension,  Air  Force-Wide 
Technical  Knowledge,  and  Global  Technical  Proficiency  Rating  Forms 


WTPT 

Measures 

Rating  Form 

Task 

Dimension 

Air  Force 

Global 

TOTWTPT 

.149** 

.171** 

.162** 

.094* 

Hands-on 

.123** 

.135** 

.155** 

.110* 

Interview 

.109* 

.122** 

.107* 

.053 

Not*.  N  ranged  from  86  to  90.  Abbreviation:  TOTWTPT,  Total  score  obtained  with 
Walk-Through  Performance  Testing. 

*p  <  .05. 


**p  <  .01. 
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Table  B-3.  Information  Systems  Radio  Operators  (AFS  492X1):  Squared 
Multiple  Correlation  Coefficients  for  Walk-Through  Performance  Test 
Measures  Using  the  Composite  Ratings  of  Self,  Peer,  and  Supervisor 
Sources  as  Predictors  Obtained  with  Task,  Dimension,  Air  Force-Wide 
Technical  Knowledge,  and  Global  Technical  Proficiency  Rating  Forms 


WTPT 

Measures 

Rating  Form 

Task 

Dimension 

Air  Force 

Global 

TOTWTPT 

.080* 

.146* 

.132* 

.130* 

Hands-on 

.076* 

.136* 

.122* 

.132* 

Interview 

.109* 

.160* 

.194* 

.159* 

Note.  N  ranged  from  142  to  148.  Abbreviation:  TOTWTPT,  Total  score  obtained 
with  Walk-Through  Performance  Testing. 

*p  <  .01. 


Table  B-4.  Jet  Engine  Mechanics  (AFS  426X2):  Squared  Multiple 
Correlation  Coefficients  for  Walk-Through  Performance  Test  Measures 
Using  the  Composite  Ratings  of  Seif,  Peer,  and  Supervisor  Sources  as 
Predictors  Obtained  with  Task,  Dimension,  Air  Force-Wide  Technical 
Knowledge,  and  Global  Technical  Proficiency  Rating  Forms 


WTPT 

Measures 

Rating  Form 

Task 

Dimension 

Air  Force 

Global 

TOTWTPT 

.132* 

.185* 

.137* 

.141* 

Hands-on 

.104* 

.177* 

.132* 

.152* 

Interview 

.106* 

.097* 

.069* 

.059* 

Note.  N 

ranged  from  217  to  222.  Abbreviation: 

TOTWTPT,  Total  score  obtained 

with  Walk-Through  Performance  Testing. 

*p  <  .01. 
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Table  B-5.  Aerospace  Ground  Equipment  Mechanics  (AFS  423X5): 

Squared  Multiple  Correlation  Coefficients  for  Walk-Through 
Performance  Test  Measures  Using  the  Composite  Ratings  of  Self, 
Peer,  and  Supervisor  Sources  as  Predictors  Obtained  with  Task, 
Dimension,  Air  Force-Wide  Technical  Knowledge,  and 
Global  Technical  Proficiency  Rating  Forms 


WTPT 

Measures 

Rating 

Form 

Task 

Dimension 

Air  Force 

Global 

TOTWTPT 

.186* 

.184* 

.163* 

.194* 

Hands-on 

.159* 

.162* 

.137* 

.165* 

Interview 

.178* 

.177* 

.168* 

.189* 

Note.  N  =  257. 

Abbreviation: 

TOTWTPT,  Total  score  obtained  with 

Walk-Through 

Performance  Testing. 
*p  <  .01. 


Table  B-6.  Aircrew  Life  Support  Specialists  (AFS  122X0):  Squared 
Multiple  Correlation  Coefficients  for  Walk-Through  Performance  Test 
Measures  Using  the  Composite  Ratings  of  Self,  Peer,  and  Supervisor 
Sources  as  Predictors  Obtained  with  Task,  Dimension,  Air  Force-Wide 
Technical  Knowledge,  and  Global  Technical  Proficiency  Rating  Forms 


WTPT 

Measures 

Rating  Form 

Task 

Dimension 

Air  Force 

Global 

TOTWTPT 

.070** 

.053* 

.018 

.047* 

Hands-on 

.070** 

.053* 

.018 

.047* 

Interview 

.126** 

.100** 

.021 

.036 

Nott.  N  ranged  from  184  to  185.  Abbreviation:  TOTWTPT,  Total  score  obtained 
with  Walk-Through  Performance  Testing. 

*p  <  .05. 

**p  <  .01. 
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Table  B-7.  Personnel  Specialists  (AFS  732X0):  Squared  Multiple 
Correlation  Coefficients  for  Walk-Through  Performance  Test  Measures 
Using  the  Composite  Ratings  of  Self,  Peer,  and  Supervisor  Sources  as 
Predictors  Obtained  with  Task,  Dimension,  Air  Force-Wide  Technical 
Knowledge,  and  Global  Technical  Proficiency  Rating  Forms 


WTPT 

Measures 

Rating 

Form 

Task 

Dimension 

Air  Force 

Global 

TOTWTPT 

.184* 

.156* 

.091* 

.160* 

Hands-on 

.184* 

.090* 

.059* 

.111* 

Interview 

.072* 

.114* 

.068* 

.104* 

Note.  N  >  190. 
Performance  Testing. 

Abbreviation: 

TOTWTPT,  Total  score  obtained  with 

Walk-Through 

*e  <  .01. 


Table  B-8.  Precision  Measurement  Equipment  Laboratory  Specialists 
(AFS  324X0):  Squared  Multiple  Correlation  Coefficients  for  Walk-Through 
Performance  Test  Measures  Using  the  Composite  Ratings  of  Self, 
Peer,  and  Supervisor  Sources  as  Predictors  Obtained  with  Task, 
Dimension,  Air  Force-Wide  Technical  Knowledge,  and 
Global  Technical  Proficiency  Rating  Forms 


Rating  Form 

WTPT  - 


Measures 

Task 

Dimension 

Air  Force 

Global 

TOTWTPT 

.195** 

.182** 

.155** 

.100** 

Hands-on 

.197** 

.178** 

.155** 

.099** 

Interview 

.071* 

.110** 

.140** 

.051 

Note.  N  ranged  from  136  to  138.  Abbreviation:  TOTWTPT,  Total  score  obtained 
with  Walk-Through  Performance  Testing. 

<  .05. 

**e  <  .01. 
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APPENDIX  C:  MEAN 
sources  USING  FI 


COMI 

SHEF 


Table  C-1.  Air  Traffic  Control  Operators  (AFS  272X0):  Number  of  Significant 
Mean  Comparisons  Among  Rater  Sources  Using  Fisher’s  Post  Hoc  LSD  Test 


Mean  Comparison* 

Rating  Form  SE>  PE  SE>  SU  PE>  SU  PE>  SE  SU>  SE  SU>  PE 


Task 

1(1) 

6(2) 

- 

- 

- 

- 

Dimension 

2(1) 

4(4) 

4(3) 

- 

- 

- 

Air  Force 

6(6) 

8(8) 

- 

- 

- 

Global 

1(1) 

2(2) 

1(1) 

- 

- 

Holt.  Number  of  tasks  =  15;  Task  dimensions  =  4;  Air  Force  wide  dimer, oions  =  8;  Global 

dimensions  =  2.  Rater  source  abbreviations:  SE,  Seif;  PE,  Peer;  SU,  Supervisor. 

‘Table  entries  indicate  the  number  of  comparisons  significant  at  p  <  .017  and  p  <  .003  (in 
parentheses) . 


Table  C-2.  Avionic  Communications  Specialists  (AFS  328X0):  Number  of 
Significant  Mean  Comparisons  Among  Rater  Sources  Using 
Fisher’s  Post  Hoc  LSD  Tests 


Mean 

Comparison3 

Rating  Form 

SE>  PE 

SE>  SU 

PE>  SU  PE>  SE  SU>  SE  SU>  PE 

Task 

Dimension 

Air  Force 
Global 

2(1) 

2(1) 

3(1) 

3(1). 

1(0) 

1(1) 

1(0) 

Note.  Number  of  tasks  -  25;  Task  dimensions  =  5;  Air  Force  wide  dimensions  =  8;  Global 
dimensions  =  2.  Rater  source  abbreviations:  SE,  Seif;  PE,  Peer,  SU,  Supervisor. 

“Table  entries  indicate  the  number  of  comparisons  significant  at  p  <  .017  and  p  <  .003  (in 
parentheses) . 


Table  C-3.  Information  Systems  Radio  Operators  (AFS  492X1):  Number  of 
Significant  Mean  Comparisons  Among  Rater  Sources  Using 
Fisher’s  Post  Hoc  LSD  Tests 


Mean  Comparison* 

Rating  Form 

SE>  PE 

SE>  SU 

PE>  SU  PE>  SE 

SU>  SE  SU>  PE 

Task 

Dimension 

Air  Force 
Global 

2(1) 

2(1) 

1(0) 

- 

2(1) 

Note.  Number  of  tasks  =  14;  Task  dimensions  =  5;  Air  Force  wide  dimensions  =  8;  Global 
dimensions  *  2.  Rater  source  abbreviations:  SE,  Self;  PE,  Peer  SU,  Supervisor. 

•Table  entries  indicate  the  number  of  comparisons  significant  at  p  <  .017  and  p  <  .003  (in 
parentheses). 
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Table  C-4.  Jet  Engine  Mechanics  (AFS  426X2):  Number  of  Significant  Mean 
Comparisons  Among  Rater  Sources  Using  Fisher’s  Post  Hoc  LSD  Tests 


Mean  Comparison* 


Rating  Form  SE>  PE  SE>  SU  PE>  SU  PE>  SE  SU>  SE  SU>  PE 


Task 

3(2) 

8(7) 

- 

• 

- 

. 

Dimension 

2(1) 

4(3) 

- 

- 

- 

- 

Air  Force 

6(5) 

5(3) 

- 

- 

- 

- 

Global 

2(2) 

2(2) 

- 

- 

- 

- 

Note.  Number  of  tasks  =  12;  Task  dimensiona  =  4;  Air  Force  wide  dimensions  =  8;  Global 
dimensions  =  2.  Rater  source  abbreviations:  SE,  Self,  PE,  Peer,  SU,  Supervisor. 

“Table  entries  indicate  the  number  of  comparisons  significant  at  p  <  .017  and  p  <  .003  (in 
parentheses) . 


Tabte  C  5.  Aerospace  Ground  Equipment  Mechanics  (AFS  423X5):  Number  of 
Significant  M-an  Comparisons  Among  Rater  Sources  Using 
Fisher's  Post  Hoc  LSD  Tests 


Mean  Comparison* 

Rating  Form 

SE>  PE 

SE>  SU 

PE>  SU  PE>  SE 

SU>  SE  SU>  PE 

Task 

Dimension 

Air  Force 
Global 

20(12) 

2(2) 

5(3) 

1(1) 

24(17) 

8(7) 

7(6) 

2(2) 

9(5) 

7(5) 

2(1) 

1(0) 

1(0) 

Note.  Number  of  tasks  = 
dimensions  =  2.  Ftater  source 
“Table  entries  indicate  the 
parentheses) . 

39;  Task  dimensions  =  9;  Air  Force  wide  dimensions  =  8;  Global 
abbreviations:  SE,  Self  PE,  Peer  SU,  Supervisor, 
number  of  comparisons  significant  at  p  <  .017  and  p  <  .003  (in 

Table  C-6.  Aircrew  Life  Support  Specialists  (AFS  122X0):  Number  of  Siqnificant 
Mean  Comparisons  Among  Rater  Sources  Using  Fisher’s  Post  Hoc  LSD  Tests 

Mean  Comparison* 

Rating  Form 

SE>  PE 

SE>  SU 

PE>  SU  PE>  SE 

SU>  SE  SU>  PE 

Task 

Dimension 

Air  Force 
Global 

8(4) 

2(2) 

4(4) 

2(2) 

9(8) 

3(2) 

4(4) 

2(2) 

- 

- 

Note.  Number  of  tasks  =  16;  Task  dimensions  =  6;  Air  Force  wide  dimensions  =  8;  Global 
dimensions  =  2.  Rater  source  abbreviations:  SE,  Seif;  PE,  Peer  SU,  Supervisor, 

“Table  entries  indicate  the  number  of  comparisons  significant  at  p  <  .017  and  p  <  .003  (in 
parentheses). 
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Table  C-7,  Personnel  Specialists  (AFS  732X0):  Number  of  Significant  Mean 
Comparisons  Among  Rater  Sources  Using  Fisher’s  Post  Hoc  LSD  Tests 


Rating  Form 

Mean  Comparison* 

SE>  PE 

SE>  SU 

PE>  SU  PE>  SE  SU>  SE  SU>  PE 

Task 

7(5) 

6(6) 

1(0) 

Dimension 

5(4) 

6(6) 

2(2) 

Air  Force 

8(7) 

8(6) 

2(0) 

Global 

2(2) 

2(2) 

1(0) 

Nof .  Number  of  tasks  =  8;  Task  dimensions  =  6;  Air  Force  wide  dimensions  =  8;  Global 

dimensions  =  2.  Rater  source  abbreviations:  SE,  Self;  PE,  Peer;  SU,  Supervisor. 


‘Table  entries  indicate  the  number  of  comparisons  significant  at  p  <  .017  and  p  <  .003  (in 
parentheses). 


Table  C-8.  Precision  Measurement  Equipment  Laboratory  Specialists 
(AFS  324X0):  Number  of  Significant  Mean  Comparisons  Among  Rater 
Sources  Using  Fisher’s  Post  Hoc  LSD  Tests 


Rating  Form 

Mean 

Comparison* 

SE>  PE 

SE>  SU 

PE>  SU  PE>  SE 

SU>  SE 

SU>  PE 

Task 

8(1) 

11(7) 

2(2) 

- 

- 

- 

Dimension 

1(0) 

4(1) 

- 

- 

- 

- 

Air  Force 

10) 

4(3) 

1(0) 

- 

- 

- 

Global 

- 

1(1) 

- 

- 

- 

- 

Note.  Number  of  tasks  » 

28;  Task  dimensions  =  ! 

5;  Air  Force  wide 

dimensions 

=  8;  Global 

dimensions  »  2. 

Rater  source 

abbreviations: 

SE,  Self;  PE,  Peer;  SU,  Supervisor. 

‘Table  entries 

indicate  the 

number  of  comparisons  significant  at  p  < 

.017  and  p 

<  .003  (in 

parentheses). 
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APPENDIX  D:  MULTITRAIT-MULTI  METHOD  ANALYSES  OF  RATING  FORMS 
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Table  D-1.  Air  Traffic  Control  Operators  (AFS  272X0):  Multitrait- 
Multimethod  Analysis  of  Variance  of  Task  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

180 

13.584 

32.749* 

.293 

.259 

1  x  S 

360 

5.772 

13.914* 

.357 

.316 

1  x  T 

2520 

.608 

1.465* 

.064 

.057 

Error 

5040 

.415 

.415 

Note 

Number  of  job  incumbents  =  181. 

Number  of  rater  sources  = 

3.  Number 

of  traits  = 

15.  VC, 

Variance 

component;  ICC,  Intraclass 

correlation 

coefficient;  1,  Job  incumbents; 

1  x  S,  Job 

incumbents 

by  rater 

sources; 

1  x  T,  Job  incumbents  by 

traits. 

•p  <  .01. 


Table  D-2.  Avionic  Communications  Specialists  (AFS  328X0):  Multitrait 
Multimethod  Analysis  of  Variance  of  Task  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

77 

20.288 

45.527* 

.265 

.236 

1  x  S 

154 

8.099 

18.175* 

.306 

.274 

1  x  T 

1848 

.754 

1.692* 

.103 

.092 

Error 

3696 

.446 

.446 

Not*.  Number  of  job  incumbents  =  78.  Number  of  rater  sources  =  3.  Number  of  traits  =  25.  VC, 
Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 


Table  D-3.  Information  Systems  Radio  Operators  (AFS  492X1):  Multitrait- 
Multimethod  Analysis  of  Variance  of  Task  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

140 

12.697 

27.106* 

.291 

.261 

1  x  S 

280 

4.965 

10.599* 

.321 

.288 

1  x  T 

1820 

.576 

1.231* 

.036 

.032 

Error 

3640 

.468 

.468 

Not*.  Number  of  iob  incumbents  »  141. 
Vanance  component;  ICC,  Intraclass  correlation 
sources;  1  x  T,  Job  incumbents  by  trauts. 

Number 

coefficient; 

of  rater  sources  = 

1,  Job  incumbents; 

3.  Number 

1  x  S,  Job 

of  traits  =  14.  VC. 
incumbents  by  rater 

*p  <  .01. 
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Table  D-4.  Jet  Engine  Mechanics  (AFS  426X2):  Multitrait- 
Multimethod  Analysis  of  Variance  of  Task  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

163 

8.342 

17.679* 

.219 

.195 

1  x  S 

326 

4.239 

8.982* 

.314 

.280 

1  x  T 

1793 

.820 

1.738* 

.116 

.104 

Error 

3586 

.472 

.472 

Note. 

Number  of 

job  incumbents  *  164. 

Number  of  rater  sources  »  3. 

Number  of  traits  = 

12.  VC, 

Variance  component;  ICC,  Intraclisa  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources-  I  «•  T,  Job  incumbents  by  traits. 

* p  <  .01. 


Table  D-5.  Aerospace  Ground  Equipment  Mechanics  (AFS  423X5):  Multitrait- 
Multimethod  Analysis  of  Variance  of  Task  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

241 

28.288 

59.372* 

.228 

.214 

1  x  S 

482 

12.826 

26.920* 

.317 

.285 

1  x  T 

9 1  38 

.719 

1.510* 

.081 

.073 

Error 

18316 

.476 

.476 

Note.  Number  of  job  incumbents  *  242.  Number  of  rater  sources  »  3.  Number  of  traits  =  39.  VC. 
Variance  component;  ICC,  Intraclass  correlation  coefficient  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 


Table  D-6.  Aircrew  Life  Support  Specialists  (AFS  122X0):  Multitrait- 
Multimethod  Analysis  of  Variance  of  Task  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

mm- 

11.070 

24.659* 

.221 

.194 

lH  m 

6.529 

14.543* 

.380 

.334 

1  x  T 

.712 

1.585* 

.088 

.077 

Error 

.449 

’49 

Note.  Number  o;  ,  b  incumbents  »  179.  Number  of  rater  sources  =  3.  Number  of  traits  =  16.  VC, 
Variance  component;  ICC,  intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  bv  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

•p  .<  01. 
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Table  D-7.  Personnel  Specialists  (AFS  732X0):  Multitrait-Multimethod 
Analysis  of  Variance  of  Task  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

186 

4.572 

7.930* 

.166 

.151 

1  x  S 

372 

2.644 

4.586* 

.258 

.234 

1  x  T 

1302 

.885 

1.535* 

.104 

.093 

Error 

2604 

.577 

.577 

Note. 

Number  of  job 

incumbents  =  187. 

Number  of  rater  sources  *  3. 

Number  of  traits 

«  8.  VC, 

Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  Incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 


Table  D-8.  Precision  Measurement  Equipment  Laboratory  Specialists  (AFS  324X0): 
Multitrait-Multimethod  Analysis  of  Variance  of  Task  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

133 

20.159 

39.091* 

.234 

.212 

1  x  S 

266 

8.358 

16.208* 

.280 

.254 

1  x  T 

3591 

.737 

1.430* 

.074 

.067 

Error 

7182 

.516 

.516 

Note.  Number  of  job  incumbents  »  134.  Number  of  rater  sources  »  3.  Number  of  traits  »  28.  VC, 
Variance  component  ICC,  Intraclass  correlation  coefficient  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

•p  <  .01. 


Table  D-9.  Air  Traffic  Control  Operators  (AFS  272X0):  Multitrait-Multimethod 
Analysis  of  Variance  of  Dimension  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

182 

4.390 

10.129* 

.330 

.295 

1  x  S 

364 

1.549 

3.574* 

.279 

.250 

1  x  T 

546 

.659 

1.520* 

.075 

.067 

Error 

1092 

.433 

.433 

Note.  Number  of  job  incumbents  »  183.  Number  of  rater  sources  *  3.  Number  of  traits  =  4,  VC. 
Variance  component;  ICC,  Intrac/ass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits, 

*p  <  .01. 


Table  D-10.  Avionic  Communications  Specialists  (AFS  328X0):  Multitrait-Multimethod 
Analysis  of  Variance  of  Dimension  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

83 

5.229 

10.117* 

.314 

.286 

1  x  S 

166 

1.784 

3.452* 

.254 

.230 

1  x  T 

332 

.562 

1.088 

.015 

.014 

Error 

664 

.517 

.517 

Note. 

Number  of  job 

incumbents  *  84. 

Number  of  rater  sources  =  3. 

Number  of  traits  = 

5.  VC, 

Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 


Table  D-1 1.  Information  Systems  Radio  Operators  (AFS  482X1):  Multitrait- 
Multimethod  Analysis  of  Variance  of  Dimension  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

140 

5.844 

13.158* 

.360 

.326 

l  x  S 

280 

1.868 

4.207* 

.285 

.258 

1  x  T 

560 

.494 

1.112 

.016 

.015 

Error 

1120 

.444 

.444 

Note.  Number  of  job  incumbents  »  141.  Number  of  rater  sources  »  3.  Number  of  traits  =  5.  VC, 
Variance  component;  ICC,  Intraclass  correlation  coefficient;  1,  Job  incumbents;  I  x  S.  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 


*p  <  .01. 


Table  D-1 2.  Jet  Engine  Mechanics  (AFS  426X2):  Multitrait-Multimethod 
Analysis  of  Variance  of  Dimension  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

214 

5.038 

12.661* 

.387 

.350 

1  x  S 

428 

1.414 

3.553* 

.254 

.230 

1  x  T 

642 

.601 

1.511* 

,068 

.061 

Error 

1284 

.398 

.398 

Note.  Number  of  job  incumbents  =  215.  Number  of  rater  sources  *  3.  Number  of  traits  =  4.  VC, 
Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 


•p  <  .01. 
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Table  D-13.  Aerospace  Ground  Equipment  Mechanics  (AFS  423X5):  Muttitrait- 
Muftimethod  Analysis  of  Variance  of  Dimension  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

254 

8.395 

18.219* 

.294 

.266 

1  x  S 

508 

2.888 

6.267* 

.270 

.245 

1  x  T 

2032 

.696 

1.509* 

.078 

.071 

Error 

4064 

.461 

.461 

Note.  Number  of  job  incumbents  =  255.  Number  of  rater  sources  =  3.  Number  of  traits  =  9.  VC, 
Variance  component:  ICC,  Intraclasa  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

«p  <  .01. 


Table  D-14.  Aircrew  Life  Support  Specialists  (AFS  122X0):  Multitrait-Multimethod 
Analysis  of  Variance  of  Dimension  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

179 

5.188 

11.033* 

.262 

.234 

1  x  S 

358 

2.295 

4.882* 

.304 

.271 

1  x  T 

895 

.724 

1.540* 

.085 

.075 

Error 

1790 

.470 

.470 

Note.  Number  of  job  incumbents  *  180.  Number  of  rater  sources  »  3.  Number  of  traits  =  6.  VC, 
Variance  component;  ICC,  Intracfaas  correlation  coefficient;  I,  Job  Incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 


Table  D-15. 

Personnel  Specialists  (AFS  732X0): 
Analysis  of  Variance  of  Dimension 

M  ultitr  ait-M  u  ttimethod 
Ratings 

Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

187 

4.480 

7.965* 

.218 

.197 

1  x  S 

374 

2.198 

3.907* 

.272 

.247 

1  x  T 

935 

.719 

1.279* 

.052 

.047 

Error 

1870 

.562 

.562 

Note.  Number  of  job  incumbents  =  188.  Number  of  rater  sources  -  3.  Number  of  traits  =  6.  VC, 
Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 
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Table  D-16.  Precision  Measurement  Equipment  Laboratory  Specialists  (AFS  324X0): 
Multitrait-Multimethod  Analysis  of  Variance  of  Dimension  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

133 

4.764 

9.441* 

.284 

.256 

1  x  S 

266 

1.937 

3.839* 

.286 

.258 

1  x  T 

532 

.609 

1.208 

.035 

.032 

Error 

1064 

.505 

.505 

Note. 

Number  of  job 

incumbents  »  134. 

Number  of  rater  sources  >  3. 

Number  of  traits  = 

5.  VC, 

Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  Incumbents  by  traits. 

*p  <  .01. 


Table  D-17.  Air  Traffic  Control  Operators  (AFS  272X0):  Multitrait-Multimethod 
Analysis  of  Variance  of  Air  Force-Wide  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

175 

5.738 

11.768* 

.219 

.194 

l  x  S 

350 

3.078 

6.312* 

.324 

.288 

1  x  T 

1225 

.774 

1.588* 

.096 

.085 

Error 

2450 

.488 

.488 

Note.  Number  of  job  incumbents  a  176.  Number  of  rater  sources  =  3.  Number  of  traits  =  8.  VC. 
Variance  component;  ICC,  Intraciass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 


Table  D-18.  Avionic  Communications  Specialists  (AFS  328X0):  Multitrait-Multimethod 
Analysis  of  Variance  of  Air  Force-Wide  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

85 

6.192 

13.596* 

.239 

.213 

1  x  S 

170 

2.665 

5.851* 

.276 

.246 

1  x  T 

595 

.912 

2.002* 

.152 

.136 

Error 

1190 

.456 

.456 

Note.  Number  of  job  incumbents  *  86.  Number  of  rater  sources  =  3.  Number  of  traits  =  8.  VC. 
Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 
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Table  D-19.  Information  Systems  Radio  Operators  (AFS  492X1):  Multitrait* 
Multimethod  Analysis  of  Variance  of  Air  Force-Wide  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

140 

5.906 

12.539* 

.226 

.200 

1  x  S 

280 

3.121 

6.628* 

.331 

.293 

1  x  T 

980 

.776 

1.647* 

.102 

.090 

Error 

1960 

.471 

.471 

Note. 

Number  of  job  incumbents  >  141. 

Number  of  rater  sources  •  3. 

Number  of  traits 

=»  8.  VC, 

Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 


Table  0-20.  Jet  Engine  Mechanics  (AFS  426X2):  Multitrait-Multimethod 
Analysis  of  Variance  of  Air  Force-Wide  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

215 

7.510 

17.784* 

.295 

.265 

1  x  S 

430 

2.740 

6.488* 

.290 

.260 

1  x  T 

1505 

.744 

1.763* 

.107 

.096 

Error 

3010 

.422 

.422 

Note.  Number  of  job  incumbents  =»  216.  Number  of  rater  sources  *  3.  Number  of  traits  =  8.  VC, 
Variance  component;  ICC,  Intraclass  correlation  coefficient  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 


Table  D-21.  Aerospace  Ground  Equipment  Mechanics  (AFS  423X5):  Multitrait- 
Multimethod  Analysis  of  Variance  of  Air  Force-Wide  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

246 

7.619 

16.365* 

.298 

.272 

1  x  S 

492 

2.396 

5.146* 

.241 

.220 

1  x  T 

1722 

.738 

1.586* 

.091 

.083 

Error 

3444 

.466 

.466 

Note.  Number  of  job  incumbents  =  247.  Number  of  rater  sources  *  3.  Number  of  traits  =  8.  VC, 
Variance  component;  ICC,  Intraclass  correlation  coefficient  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 
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Table  D-22.  Aircrew  Life  Support  Specialists  (AFS  122X0):  Multitrait- 
Multimethod  Analysis  of  Variance  of  Air  Force-Wide  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

178 

7.513 

17.307* 

.295 

.265 

1  x  S 

356 

2.714 

6.253* 

.285 

.256 

1  x  T 

1246 

.731 

1.684* 

.099 

.089 

Error 

2492 

.434 

.434 

Note.  Number  of  job  incumbents  »  179.  Number  of  rater  sources  =  3.  Number  of  traits  =  8.  VC, 
Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 


Table  D-23.  Personnel  Specialists  (AFS  722X0):  Multitrait-Multimethod 
Analysis  of  Variance  of  Air  Force-Wide  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

185 

5.963 

12.750* 

.229 

.204 

1  x  S 

370 

3.008 

6.432* 

.318 

.282 

1  x  T 

1295 

.800 

1.711* 

.111 

.099 

Error 

2590 

.468 

.468 

Note.  Number  of  job  incumbents  *  186.  Number  of  rater  sources  =  3.  Number  of  traits  =  8.  VC, 
Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 


Table  D-24.  Precision  Measurement  Equipment  Laboratory  Specialists  (AFS  324X0): 
Muttitrait-Muitimethod  Analysis  of  Variance  of  Air  Force-Wide  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

134 

6.454 

13.678* 

.249 

.225 

1  x  S 

268 

2.533 

5.368* 

.258 

.232 

1  x  T 

938 

.864 

1.832* 

.131 

.118 

Error 

1876 

.472 

.472 

Note.  Number  of  job  incumbents  *  135.  Number  of  rater  sources  =  3.  Number  of  traits  =  8.  VC, 
Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  01 
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Table  D-25.  Air  Traffic  Control  Operators  (AFS  272X0):  Multitrait-Multimethod 
Analysis  of  Variance  of  Global  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

183 

2.350 

5.439* 

.320 

.276 

1  x  S 

366 

1.038 

2.401* 

.303 

.261 

1  x  T 

183 

.743 

1.720* 

.104 

.090 

Error 

366 

.432 

.432 

Not*.  Number  of  job  incumbents  =  184.  Number  of  rater  sources  *  3.  Number  of  traits  »  2.  VC, 
Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 


*p  <  .01. 


Table  D-26.  Avionic  Communications  Specialists  (AFS  328X0):  Multitrait- 
Multimethod  Analysis  of  Variance  of  Global  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

85 

2.035 

3.708* 

.248 

.212 

1  x  S 

170 

.905 

1.648* 

.178 

.152 

1  x  T 

85 

1.128 

2.056* 

.193 

.165 

Error 

170 

.549 

.549 

Note.  Number  of  job  incumbents  =  86.  Number  of  rater  sources  ■  3.  Number  of  traits  =  2.  VC, 
Variance  component;  ICC,  Intraclaas  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T.  Job  incumbents  by  traits. 

*p  <  .01. 


Table  D-27.  Information  Systems  Radio  Operators  (AFS  492X1):  Multitrait- 
Multimethod  Analysis  of  Variance  of  Global  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

143 

2.246 

4.597* 

.293 

.250 

1  x  S 

286 

.862 

1.764* 

.187 

.159 

1  x  T 

143 

1.096 

2.244* 

.202 

.173 

Error 

286 

.488 

.488 

Not*.  Number  of  iob  incumbents  =  144.  Number 
Variance  component;  ICC,  Intraclass  correlation  coefficient; 
sources;  1  x  T,  Job  incumbents  by  traits. 

of  rater  sources  * 
1,  Job  incumbents; 

3.  Number  of  traits  = 
1  x  S,  Job  incumbents 

2.  VC, 

by  rater 

*p  <  .01. 
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Table  D-28.  Jet  Engine  Mechanics  (AFS  426X2):  Multitrait-Multimethod  Analysis 

of  Variance  of  Global  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

217 

2.386 

5.601* 

.327 

.280 

1  x  S 

434 

.965 

2.266* 

.270 

.231 

1  x  T 

217 

.859 

2.018* 

.144 

.124 

Error 

434 

.426 

.426 

Note. 

Number  of  job 

incumbents  >  218. 

Number  of  rater  sources  «  3. 

Number  of  traits 

=  2.  VC, 

Variance  component;  ICC,  Intraclass  con-elation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

•p  <  .01. 


Table  D-29.  Aerospace  Ground  Equipment  Mechanics  (AFS  423X5):  Muititrait- 
Muitimethod  Analysis  of  Variance  of  Global  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

256 

''.635 

6.939* 

.376 

.328 

1  x  S 

512 

1.017 

2.678* 

.319 

.278 

1  x  T 

256 

.593 

1.560* 

.071 

.062 

Error 

512 

.380 

.380 

Note.  Number  of  job  incumbents  «  257.  Number  of  rater  sources  =  3.  Number  of  traits  =  2.  VC, 
Variance  component;  ICC.  Intraclasa  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 


Table  D-30.  Aircrew  Life  Support  Specialists  (AFS  122X0):  Muititrait-Muitimethod 
Analysis  of  Variance  of  Global  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

223 

2.407 

5.320* 

.326 

.290 

1  x  S 

446 

1.085 

2.398* 

.316 

.281 

1  x  T 

223 

.545 

1.204 

.031 

.027 

Error 

446 

.452 

.452 

Note.  Number  of  job  incumbents  *  224.  Number  of  ra'er  sources  =  3.  Number  of  traits  =  2.  VC, 
Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T.  Job  incumbents  by  traits. 

•p  <  .01. 
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Table  D-31.  Personnel  Specialists  (AFS  732X0):  Multitrait-Muitimethod 
Analysis  of  Variance  of  Global  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

210 

2.026 

4.026* 

.254 

.216 

1  x  S 

420 

.955 

1.898* 

.226 

.192 

1  x  T 

210 

1.086 

2.158* 

.194 

.165 

Error 

420 

.503 

.503 

Note.  Number  of  job  incumbents  =  211.  Number  of  rater  sources  =  3.  Number  of  traits  =  2.  VC, 
Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

*p  <  .01. 


Table  D-32.  Precision  Measurement  Equipment  Laboratory  Specialists  (AFS  324X0): 
Multitrait-Multimethod  Analysis  of  Variance  of  Global  Ratings 


Source 

df 

MS 

F-Ratio 

VC 

ICC 

1 

167 

2.220 

4.892* 

.294 

.250 

1  x  S 

334 

.988 

2.177* 

.267 

.227 

1  x  T 

167 

.933 

2.057* 

.160 

.136 

Error 

334 

.454 

.454 

Note.  Number  of  job  incumbents  ■  168.  Number  of  rater  sources  =  3.  Number  of  traits  =  2.  VC, 
Variance  component;  ICC,  Intraclass  correlation  coefficient;  I,  Job  incumbents;  I  x  S,  Job  incumbents  by  rater 
sources;  I  x  T,  Job  incumbents  by  traits. 

•p  <  .01. 
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aPHtNOIX  E:  CORRELATIONS  BETWEEN  ASVAB  PREDICTORS  AND 
JOB  PERFORMANCE  MEASURES,  FACTORS,  AND  TRAINING 
SCHOOL  GRADES  (UNCORRECTED  FOR  RESTRICTION) 
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Table  E-1.  Air  Traffic  Control  Operators  (AFS  272X0):  Correlations  Between  ASVAB 
Predictors  and  Job  Performance  Measures,  Factors,  and  Training  School  Grades 

(Uncorrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

M 

A 

G* 

E 

Subtestsb 

TECH 

.106 

.113 

.053 

.128 

.082 

.276 

SUPER 

.111 

.065 

.047 

.041 

.354* 

SELF 

-.086 

-.017 

-.022 

-.093 

-.114 

.301 

PEER 

.161 

.048 

.069 

.058 

.333 

INPERS 

.016 

.035 

-.005 

-.069 

.331 

TOTWTPT 

.109 

.146 

.055 

.146 

.095 

.276 

Hands-on 

.066 

.157 

.052 

.092 

.075 

.302 

Interview 

.131 

.064 

.049 

.149 

.081 

.264 

TGRD 

.421** 

.193* 

.229* 

.416** 

.343** 

.454** 

Note.  N  =  157.  Abbreviations 
G,  General;  E,  Electronics;  TOTWTPT, 

are:  AFQT, 
Total  score 

Armed  Forces  Qualification  Test;  M,  Mechanical;  A, 
obtained  with  Walk-Through  Performance  Testing;  and 

Administrative; 
TGRD,  Grade 

received  in  technical  training. 

“Composite  used  for  selection  into  the  specialty. 
bThe  values  reported  for  the  subtests  are  multiple  correlations. 
*p  <  .05. 

**p  <  .01. 


Table  E-2.  Avionic  Communications  Specialists  (AFS  328X0):  Correlations  Between  ASVAB 
Predictors  and  Job  Performance  Measures,  Factors,  and  Training  School  Grades 

(Uncorrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

M 

A 

G 

Ea 

Subtestsb 

TECH 

.334* 

.295* 

-.015 

.296* 

.317* 

.555** 

SUPER 

.175 

.180 

.083 

.173 

.216 

.353 

SELF 

-.019 

.096 

-.038 

.041 

.048 

.363 

PEER 

.145 

.005 

.041 

.179 

.135 

.301 

INPERS 

.138 

-.053 

.130 

.179 

.065 

.414 

TOTWTPT 

.362** 

.286* 

.003 

.323* 

.308* 

.562** 

Hands-on 

.281* 

.289* 

.025 

.231 

.298* 

.454 

Interview 

.319* 

.248 

-.046 

.299* 

.277* 

.581** 

TGRD 

.532** 

.350* 

.274* 

.431** 

.543** 

.661** 

Note.  N  = 

74. 

Abbreviations 

are:  AFQT,  Armed 

Forces  Qualification  Test;  M,  Mechanical:  A. 

Administrative;  G, 

General;  E,  Electronics;  TOTWTPT, 

Total  score  obtainee 

1  with  Walk-Through  Performance 

Testing;  and  TGBD,  Grade 

received  in  technical 

training. 

Composite 

used 

for  selection 

into  the  specialty. 

bThe  values 

reported  for  the  : 

subtests  are  multiple 

correlations. 

*p  <  .05. 

**p  <  01. 
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Table  E-3.  Information  Systems  Radio  Operators  (AFS  492X1):  Correlations  Between  ASVAB 
Predictors  and  Job  Performance  Measures,  Factors,  and  Training  School  Grades 

(Uncorrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

M 

A* 

G 

E 

Subtestsb 

TECH 

.374** 

.254* 

-.049 

.370** 

.316** 

.480** 

SUPER 

.277* 

.290* 

.094 

.247* 

.284* 

.449* 

SELF 

.157 

.071 

.102 

.144 

.149 

.382 

PEER 

.360** 

.310** 

.051 

.312** 

.343** 

.434* 

INPERS 

.194 

.137 

.060 

.162 

.158 

.444* 

TOTWTPT 

.365** 

.236* 

-.047 

.364** 

.302* 

.459* 

Hands-on 

.314** 

.188 

-.073 

.329** 

.251* 

417 

Interview 

.397** 

.298* 

-.016 

.370** 

.351** 

.537** 

TGRD 

.604** 

.509** 

.228 

.607** 

.572** 

.664** 

Not*.  N  =  97.  Abbreviations  are:  AFQT,  Armed  Forces  Qualification  Test;  M,  Mechanical;  A,  Administrative;  G, 
General;  E,  Electronics;  TOTWTPT,  Total  score  obtained  with  Walk-Through  Performance  Testing;  and  TGRD,  Grade 
received  in  technical  training. 

‘Composite  used  for  selection  into  the  specialty. 

bThe  values  reported  for  the  subtests  are  multiple  correlations. 

*p  <  .05. 

“p  <  .01. 


Table  E-4.  Jet  Engine  Mechanics  (AFS  426X2):  Correlations  Between  ASVAB  Predictors  and 
Job  Performance  Measures,  Factors,  and  Training  School  Grades 
(Uncorrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

Ma 

A 

G 

E 

Subtestsb 

TECH 

.178 

.208* 

.152 

.227* 

.217* 

.348* 

SUPER 

.044 

.070 

.080 

.069 

.089 

.228 

SELF 

.040 

.086 

.057 

.007 

.118 

.334* 

PEER 

.106 

.049 

.140 

.131 

.092 

.210 

INPERS 

.076 

.026 

.132 

.088 

.065 

.205 

TOTWTPT 

.138 

.186* 

.148 

.187* 

.176 

.334* 

Hands-on 

.086 

.149 

.126 

.136 

.125 

.324 

Interview 

.214* 

.208* 

.137 

.249** 

.243** 

.310 

TGRD 

.529** 

.470** 

.392** 

.017** 

.544** 

.606** 

No**-  N  =  166,  Abbreviations  are;  AFQT,  Armed  Forces  Qualification  Test;  M,  Mechanical;  A,  Administrative. 
G,  General;  E,  Electronics;  TOTWTPT,  Total  score  obtained  with  Walk-Through  Performance  Testing;  and  TGRD.  Grade 
received  in  technical  training. 

‘Composite  used  for  selection  into  the  specialty. 

bThe  values  reported  for  the  subtests  are  multiple  correlations. 

*p  <  .05. 

**p  <  .01, 
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Table  E-S.  Aerospace  Ground  Equipment  Mechanics  (AFS  423X5):  Correlations  Between 
ASVAB  Predictors  and  Job  Performance  Measures,  Factors,  and  Training  School  Grades 

(Unccrrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

Ma 

A 

G 

E 

Subtestsb 

TECH 

.241** 

.423** 

.111 

.227** 

.365** 

.513** 

SUPER 

-.011 

.076 

.023 

-.003 

.029 

.176 

SELF 

-.069 

.239** 

.160* 

-.090 

.052 

.395** 

PEER 

-.074 

.188** 

.018 

-.027 

-.007 

.301* 

INPERS 

-.047 

.059 

.042 

-.045 

-.015 

.171 

TOTWTPT 

.154* 

.317** 

.038 

.140* 

.287** 

.438** 

Hands-on 

.191** 

.334** 

.046 

.179** 

.322** 

.463** 

Interview 

.084 

.280** 

.023 

.078 

.  1 92** 

.362** 

JKTSCORE 

.348** 

.453** 

,233** 

.332** 

.408** 

.544** 

TGRD 

.265** 

.373** 

.216** 

.255** 

.338** 

.477** 

Note.  N  =  202. 

Abbreviations 

are;  AFQT,  Armed  Forces  Qualification  Test;  M, 

Mechanical;  A, 

Administrative; 

G,  General;  E,  Electronics;  TOTWTPT,  Total  score  obtained  with  Walk-Through  Performance  Testing;  and  TGRD,  C 

received  in  technical  training. 

‘Composite  used  for  selection  into  the  specialty. 
bThe  values  reported  for  the  subtests  are  multiple  correlations. 
* p  <  .05. 

**p  <  .01. 


Table  E-6.  Aircrew  Life  Support  Specialists  (AFS  122X0):  Correlations  Between  ASVAB 
Predictors  and  Job  Performance  Measures,  Factors,  and  Training  School  Grades 

(Uncorrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

M 

A 

Ga 

E 

Subtestsb 

TECH 

.146 

.272** 

-.046 

.162* 

.180* 

.317 

SUPER 

.136 

.098 

.042 

.091 

.091 

.239 

SELF 

-.108 

-.015 

-.132 

-.146 

-.088 

.262 

PEER 

.117 

.045 

-.115 

.055 

.096 

.313 

INPERS 

.062 

.016 

-.109 

-.023 

.063 

.304 

TOTWTPT 

.088 

.167* 

-.052 

.099 

.130 

.206 

Hands-on 

.088 

.167* 

-.052 

.099 

.130 

.206 

Interview 

.096 

.261** 

-.113 

.096 

.152 

.334 

JKTSCORE 

.175* 

.270** 

.023 

.199** 

.169* 

.363* 

TGRD 

.458** 

.312** 

.1/9* 

.452** 

.408** 

.486** 

Note  N  = 

157, 

Abbreviations 

are;  AFQT,  Armed  Forces  Qualification  Test;  M, 

Mechanical;  A,  Administrative: 

G,  General;  E,  Electronics;  TOTWTPT,  Total  score  obtained  with  Walk-Through  Performance  Testing;  and  TGRD,  Grade 

received  in  technical 

training. 

Composite 

used 

for  selection  into  the  specialty. 

bThe  values 

reported  for  the  subtests  are  multiple  correlations. 

*p  <  .05. 

**p  <  01 , 
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Table  F-7.  Personnel  Specialists  (AFS  732X0):  Correlations  Between  ASVAB  Predictors  and 
Job  Performance  Measures,  Factors,  and  Training  School  Grades 
(Uncorrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

M 

A* 

G 

E 

Subtests0 

TECH 

.321** 

.144 

.179* 

.326** 

.221** 

.387** 

SUPER 

.115 

.102 

.087 

.127 

.089 

.303 

SEL" 

-.044 

.019 

.145 

-.049 

-.024 

.317 

PEER 

.178* 

.087 

.174* 

.176* 

.125 

.304 

INTERS 

.108 

.098 

.092 

.131 

.089 

.248 

TOTWTPT 

.252** 

.126 

.168* 

.256** 

.164* 

.327 

Hands-on 

.245** 

.148 

.135 

.227** 

.196** 

.291 

Interview 

.185** 

.038 

.124 

.223** 

.092 

.308 

JKTSCORE 

.294** 

.126 

.137 

.290** 

.202** 

.377** 

TGRD 

.466** 

.202** 

.232** 

.472** 

.350*“ 

.510** 

Note.  N  =  167. 

Abbreviations 

are;  AFQT,  Armed  Forces  Qualification  Test;  M, 

Mechanical;  A, 

Administrative; 

G,  General;  E,  Electronics;  TOTWTPT,  Total  score  obtained  with  Walk-Through  Performance  Testing;  and  TGRD,  Grade 

received  in  technical  training. 

"Composite  used  for  selection  into  the  specialty. 

bThe  values  reported  for  the  subtests  are  multiple  correlations. 

*p  <  .05. 

**p  <  .01. 


Table  E-8.  Precision  Measurement  Equipment  Laboratory  Specialists  (AFS  324X0): 
Correlations  Between  ASVAB  Predictors  and  Job  Performance  Measures,  Factors,  ana 
Training  School  Grades  (Uncorrected  for  Restriction) 


Job 

ASVAB 

Predictors 

Performance 

Measures 

AFQT 

M 

A 

G 

E* 

Subtestsb 

TECH 

.334** 

.328** 

.226** 

.345** 

.418** 

.552** 

SUPER 

.101 

.156 

.167 

.082 

.063 

.265 

SELF 

.1J5* 

.252** 

.234** 

.142 

.126 

.503** 

PEER 

.272** 

.237** 

.364** 

.320** 

.248** 

.458** 

INPERS 

.066 

.084 

.188* 

.064 

-.034 

.309 

TOTWTPT 

.307**  ■ 

.236** 

.210** 

.310** 

.365** 

.503** 

Hands-on 

.296** 

.229** 

.206* 

.299** 

.351** 

.498** 

Interview 

.215** 

.286** 

.176* 

.254** 

.280** 

.396* 

JKTSCORE 

.306** 

.385** 

.174 

.310** 

.415** 

.560** 

TGRD 

.37  7** 

.420** 

.224** 

.309** 

.438** 

.626** 

Note,  N  =  123,  Abbreviation;  are:  AFQT.  Armed  Forces  Qualification  Test:  M.  Mechanical:  A. 
G,  General;  E,  Electronics;  TOTWTP  ',  Total  score  obtained  with  Walk-Through  Performance  Testing;  and 
received  in  technical  training. 

'Composite  used  for  selection  into  the  specialty. 

BThe  values  reported  for  the  subtests  are  multiple  correlations. 

Admmisiraiivc: 
TGRD,  Grade 

*p  <  05. 

**p  <  .01. 


APPENDIX  F:  CORRELATIONS  BETWEEN  ASVAB  PREDICTORS  AND  JOB 
PERFORMANCE  MEASURES,  FACTORS,  AND  TRAINING  SCHOOL  GRADES 

^CORRECTED  FOR  RESTRICTION) 
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Table  F-1.  Air  Traffic  Control  Operators  (AFS  272X0):  Correlations  Between  ASVAB 
Predictors  and  Job  Performance  Measures,  Factors,  and  Training  School  Grades 

(Corrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

M 

A 

Ga 

E 

Subtestsb 

TECH 

.175 

.177 

.119 

.185 

.188 

.343 

SUPER 

.062 

.096 

.041 

.065 

.077 

.395* 

SELF 

-.144 

-.108 

-.120 

-.148 

-.178 

.375 

PEER 

.117 

.174 

.085 

.126 

.133 

.386 

INPERS 

-.080 

-.059 

-.072 

-.069 

-.085 

.369 

TOTWTPT 

.214 

.235 

.152 

.230 

.232 

.360 

Hands-on 

.072 

.156 

.047 

.048 

.115 

.356 

'nterview 

.248 

.177 

.171 

.255 

.234 

.359 

TGRD 

.755** 

.546* 

.648* 

.753** 

.705** 

.768** 

Note.  Statistical  significance  reported  for  uncorrected  correlations  is  inferred  for  corrected  correlations. 

‘Composite  used  for  selection  into  the  specialty. 

bThe  values  reported  for  the  subtests  are  multiple  correlations. 

* p  <  .05. 

**p  <  .01. 


Table  F-2.  Avionic  Communications  Specialists  (AFS  328X0):  Correlations  Between  ASVAB 
Predictors  and  Job  Performance  Measures,  Factors,  and  Training  School  Grades 

(Corrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

M 

A 

G 

E* 

Subtestsb 

TECH 

.635* 

.589* 

.437 

.613* 

.655* 

.778** 

SUPER 

.391 

.389 

.297 

.381 

.431 

.570 

SELF 

-.034 

.174 

-.065 

-.003 

.061 

.469 

PEER 

.246 

.153 

.154 

.252 

.273 

.422 

INPERS 

.214 

.055 

.181 

.214 

.184 

.552 

TOTWTPT 

.668** 

.588* 

.469 

.645* 

.668* 

.789** 

Hands-on 

.612* 

.576* 

.448 

.585 

.633* 

.724 

Interview 

.580* 

.531 

.376 

.566* 

.597* 

.764** 

TGRD 

.881** 

.696* 

.754* 

.853** 

.876** 

.921** 

Not*.  Statistical  significance 
‘Composite  used  for  selection 
bThe  values  reported  for  the 

reported  for  uncorrected  correlations 
into  the  specialty, 
subtests  are  multiple  correlations. 

is  inferred  for 

corrected  correlations. 

*p  <  05. 

**p  <  01. 
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Table  F-3.  Information  Systems  Radio  Operators  (AFS  492X1):  Correlations  Between  ASVAB 
Predictors  and  Job  Performance  Measures,  Factors,  and  Training  School  Grades 

(Corrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

M 

A* 

G 

E 

Subtestsb 

TECH 

.288** 

.223* 

.056 

.298** 

.250** 

.542** 

SUPER 

.550* 

.466* 

.488 

.558* 

.516* 

.664* 

SELF 

.443 

.283 

.445 

.447 

.425 

.606 

PEER 

.463** 

.388** 

.328 

.456** 

.432** 

.543* 

INPERS 

.484 

.316 

.434 

.491 

.401 

.661* 

TOTWTPT 

.277** 

.212* 

.048 

.288** 

.246* 

.510* 

Hands-on 

.191** 

.144 

-.031 

.209** 

.171* 

.464 

'nterview 

.358** 

,280* 

.142 

.358** 

.305** 

.611** 

TGRD 

.790** 

.651** 

.658 

.788** 

.754** 

.823** 

Note.  Statistical  significance  reported  for  uncorrected  correlations  is  inferred  for  corrected  correlations. 

"Composite  used  for  selection  into  the  specialty. 

bThe  values  reported  for  the  subtests  are  multiple  correlations. 

•p  <  .05. 

**p  <  .01. 


Table  F-4.  Jet  Engine  Mechanics  (AFS  426X2):  Correlations  Between  ASVAB  Predictors  and 
Job  Performance  Measures,  Factors,  and  Training  School  Grades 
(Corrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

M* 

A 

G 

E 

Subtests1 

TECH 

.375 

.421* 

*  .312 

.402* 

.417* 

.502* 

SUPER 

.119 

.154 

.130 

.129 

.157 

.260 

SELF 

-.010 

.114 

-.044 

-.026 

.084 

.411* 

PEER 

.215 

.164 

.241 

.227 

.201 

.280 

INPERS 

.166 

.119 

.202 

.171 

.155 

.256 

TOTWTPT 

.332 

.383* 

.298 

.359* 

.369 

.472* 

Hands-on 

.229 

.306 

.218 

.257 

.272 

.420 

Interview 

.414* 

.424* 

.322 

.434** 

.446** 

.488 

TGRD 

.785** 

.736** 

.680** 

.783** 

.791** 

.830** 

Note.  Statistical  significance  reported  for  uncorrected  correlations  is  inferred  for  corrected  correlations. 

"Composite  used  for  selection  into  the  specialty. 

bThe  values  reported  for  the  subtests  arc  multiple  correlations. 

•p  <  .05. 

**p  <  .01. 
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Table  F-5.  Aerospace  Ground  Equipment  Mechanics  (AFS  423X5):  Correlations  Between 
ASVAB  Predictors  and  Job  Performance  Measures,  Factors,  and  Training  School  Grades 

(Corrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

M* 

A 

G 

E 

Subtestsb 

TECH 

.559** 

.684** 

.442 

.556** 

.647** 

.729** 

SUPER 

.041 

.128 

.028 

.046 

.084 

.223 

SELF 

.220 

.437** 

.236* 

.218 

.320 

.528** 

PEER 

.040 

.242** 

.027 

.061 

.110 

.362* 

INPERS 

-.005 

.082 

.008 

-.003 

.031 

.205 

TOTWTPT 

.352* 

.502** 

.250 

.348* 

.456** 

.588** 

Hands-on 

.404** 

.546** 

.285 

.400** 

.512** 

.632** 

Interview 

.267 

.427** 

.192 

.267 

.351** 

.489*’ 

JKTSCORE 

.736** 

.763** 

.634** 

.734** 

.771** 

.821** 

TGRD 

.654** 

.684** 

.578** 

.654** 

.696** 

.748** 

Note.  Statistical  significance  reported  for  uncorrected  correlations  is  inferred  for  corrected  correlations. 

‘Composite  used  for  selection  into  the  specialty. 

bThe  values  reported  for  the  subtests  are  multiple  correlations. 


•p  <  .05. 

"p  <  .01. 


Table  F-6.  Aircrew  Life  Support  Specialists  (AFS  122X0):  Correlations  Between  ASVAB 
Predictors  and  Job  Performance  Measures,  Factors,  and  Training  School  Grades 

(Corrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

M 

A 

Ga 

E 

Subtests1 

TECH 

.212 

-.304** 

.115 

.221* 

.252* 

.357 

SUPER 

.269 

.192 

.232 

.245 

.227 

.340 

SELF 

-.434 

-.294 

-.423 

-.449 

-.393 

.493 

PEER 

.044 

-.007 

-.075 

.016 

.025 

.333 

INPERS 

-.127 

-.115 

-.195 

-.165 

-.119 

.336 

TOTWTPT 

.145 

.202* 

.067 

.153 

.177 

.242 

Hands-on 

.145 

.202* 

.067 

.153 

.177 

.242 

Interview 

.026 

.165** 

-.071 

.027 

.080 

.318 

JKTSCORE 

.300* 

.361** 

.226 

.310** 

.323* 

.435* 

TGRD 

.706** 

.587** 

.593* 

.702** 

.689** 

.719** 

Note.  Statistical  significance  reported  for  uncorrected  correlations  is  inferred  for  corrected  correlations. 

‘Composite  used  for  selection  into  the  specialty. 

bThe  values  reported  for  the  subtests  are  multiple  correlations. 

*p  <  .05. 

-p  <  .01. 
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Table  F-7.  Personnel  Specialists  (AFS  732X0):  Correlations  Between  ASVAB  Predictors  and 
Job  Performance  Measures,  Factors,  and  Training  School  Grades 
(Corrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

M 

A* 

G 

E 

Subtestsb 

TECH 

.639** 

.408 

.583* 

.637** 

.564** 

.667** 

SUPER 

.171 

.165 

.130 

.178 

.170 

.358 

SELF 

.303 

.211 

.418 

.299 

.260 

.485 

PEER 

.399* 

.282 

.378* 

.400* 

.363 

.482 

INPERS 

.338 

.273 

.321 

.348 

.314 

.410 

TOTWTPT 

.547** 

.365 

.510* 

.547** 

.482* 

.581 

Hands-on 

.458** 

.333 

.396 

.451** 

.426** 

.481 

'nterview 

.509** 

.286 

.499 

.517** 

.429 

.569 

JKTSCORE 

.601** 

.371 

.538 

.595** 

.523** 

.638** 

TGRD 

.719** 

.472** 

.613** 

.719** 

.660** 

.738** 

Note.  Statistical  significance  reported  for  uncorrected  correlations  is  inferred  for  corrected  correlations. 

“Composite  used  for  selection  into  the  specialty. 

bThe  values  reported  for  the  subtests  are  multiple  correlations. 

•p  <  .05. 

**p  <  .01. 


Table  F-8.  Precision  Measurement  Equipment  Laboratory  Specialists  (AFS  324X0): 
Correlations  Between  ASVAB  Predictors  and  Job  Performance  Measures,  Factors,  and 
Training  School  Grades  (Corrected  for  Restriction) 


Job 

Performance 

Measures 

ASVAB 

Predictors 

AFQT 

M 

A 

G 

Ea 

Subtestsb 

TECH 

.736** 

.697** 

.600** 

.741** 

.798** 

.840** 

SUPER 

.260 

.263 

.309 

.247 

.236 

.399 

SELF 

.435* 

.409** 

.446** 

.402 

.409 

.654** 

PEER 

.727** 

.602** 

.723** 

.738** 

.706** 

.785** 

INPERS 

.176 

.134 

.292* 

.172 

.105 

.428 

TOTWTPT 

.680** 

.617** 

.552** 

.685** 

.736** 

.790** 

Hands-on 

.657** 

.605** 

.532* 

.662** 

.716** 

.778** 

Interview 

.694** 

.652** 

.604* 

.704** 

.726** 

.760* 

JKTSCORE 

.720** 

.720** 

.570 

.719** 

.789** 

.833** 

TGRD 

.806** 

.723** 

.683** 

.794** 

.814** 

.884** 

Note.  Statistical  significance  reported  for  uncorrec'ed  correlations  is  inferred  for  corrected  correlations. 

“Composite  used  for  selection  into  the  specialty. 

bThe  values  reported  for  the  subtests  are  multiple  correlations. 

•p  <  .05. 

**p  <  .01. 
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APPENDIX  G:  ROY-BARGMAN  STEP-DOWN  TESTS 
(UNCORRECTED  FOR  RESTRICTION) 
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Table  G-1.  Air  Traffic  Control  Operators  (AFS  272X0):  Summary  of  Roy-Bargman 
Step-Down  Tests  (Uncorrected  for  Restriction) 


Analysis 

df 

Mean 

Squares 

Set  Order 

HYP 

MSE 

HYP 

MSE 

F-ratio 

1  TOTWTPT 

10 

160 

673.40 

355.62 

1.89* 

TGRD 

10 

159 

116.76 

28.38 

4.11** 

2  Hands-on 

10 

160 

334.17 

160.44 

2.08* 

Interview 

10 

159 

137.62 

69.07 

1.99* 

TGRD 

10 

158 

114.37 

28.55 

4.00** 

3  TGRD 

10 

160 

121.07 

28.24 

4.29** 

TOTWTPT 

10 

159 

626.36 

57.44 

1.75 

4  TGRD 

10 

160 

121.07 

28.24 

4.29** 

Hands-on 

10 

159 

327.18 

161.44 

2.03* 

Interview 

10 

158 

126.31 

69.40 

1.82 

5  TGRD 

10 

160 

121.07 

28.24 

4.29** 

Interview 

10 

159 

337.06 

206.28 

1.63 

Hands-on 

10 

158 

120.28 

54.31 

2.21* 

Note.  Abbreviations  are: 
*p  <  .05. 

**p  <  .01. 

HYP,  Hypothesis;  and 

MSE,  Mean  square  error. 

Table  G-2.  Avionic  Communications  Specialists  (AFS  328X01:  Summary  of 
Roy-Bargman  Step-Down  Tests  (Uncorrected  for  Restriction) 

Analysis 

df 

Mean 

Squares 

F-ratio 

Set 

Order 

HYP 

MSE 

HYP 

MSE 

1 

TOTWTPT 

10 

75 

1444.18 

378.89 

3.81** 

TGRD 

10 

74 

78.24 

16.30 

4.80** 

2 

Hands-on 

10 

75 

486.90 

175.37 

2.78** 

Interview 

10 

74 

183.51 

151.45 

1.21 

TGRD 

10 

73 

79,86 

16.35 

4.88** 

3 

TGRD 

10 

75 

84,20 

16.17 

5.21** 

TOTWTPT 

10 

74 

1321.60 

381.89 

3.46** 

4 

TGRD 

10 

75 

84.20 

16.17 

5.21** 

Hands-on 

10 

74 

436.33 

177.26 

2.46* 

Interview 

10 

73 

197.73 

151.47 

1.30 

5 

TGRD 

10 

75 

84.20 

16.17 

5.21** 

Interview 

10 

74 

667.48 

236.29 

2.82** 

Hands-on 

10 

73 

113.58 

113.63 

1.00 

Nof  Abbreviations  are:  HYP,  Hypothesis;  and  MSE,  Mean  square  error 
*p  <  .05. 

**p  <  .01. 
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Table  G-3.  Information  Systems  Radio  Operators  (AFS  492X1):  Summary  of 
Roy-Bargman  Step-Down  Tests  (Uncorrected  for  Restriction) 


Analysis  df  Mean  Squares 


Set 

Order 

HYP 

MSE 

HYP 

MSE 

F-ratio 

1 

TOTWTPT 

10 

97 

1899.82 

710.76 

2.67** 

TGRD 

10 

96 

149.12 

25.63 

5.82** 

2 

Hands-on 

10 

97 

953.16 

419.61 

2.27* 

Interview 

10 

96 

327.26 

128.40 

2.55** 

TGRD 

10 

95 

139.26 

25.52 

5.46** 

3 

TGRD 

10 

97 

231.80 

28.66 

8.09** 

TOTWTPT 

10 

96 

714.90 

635.60 

1.12 

4 

TGRD 

10 

97 

231.80 

28.66 

8.09** 

Hands-on 

10 

96 

418.81 

387.77 

1.08 

Interview 

10 

95 

241.32 

132.72 

1.95* 

5 

TGRD 

10 

97 

231.80 

28.66 

8.09** 

Interview 

10 

96 

450.62 

268.32 

1.68 

Hands-on 

10 

95 

239.97 

178.79 

1.34 

Not*.  Abbreviations  are:  HYP,  Hypothesis;  and  MSE,  Mean  squa/e  error. 
*p  <  .05. 

**p  <  .01. 


Table  G-4.  Jet  Engine  Mechanics  (AFS  426X2):  Summary  of  Roy-Bargman 
Step-Down  Tests  (Uncorrected  for  Restriction) 


Analysis 

df 

Mean 

Squares 

F-ratio 

Set 

Order 

HYP 

MSE 

HYP 

MSE 

1 

TOTWTPT 

10 

177 

436.57 

224.07 

1.95* 

TGRD 

10 

176 

258.96 

29.47 

8.79** 

2 

Hands-on 

10 

177 

196.62 

105.46 

1.86 

Interview 

10 

176 

104.36 

107.65 

.97 

TGRD 

10 

175 

260.11 

29.36 

8.86** 

3 

TGRD 

10 

177 

384.30 

30.20 

9.41** 

TOTWTPT 

10 

176 

332.55 

218.69 

1.52 

4 

TGRD 

10 

177 

284.30 

30.20 

9.41** 

Hands-on 

10 

176 

177.32 

101.98 

1.74 

Interview 

10 

175 

82.50 

108.26 

.76 

5 

TGRD 

10 

177 

284.30 

30.20 

9.41** 

Interview 

10 

176 

140.08 

142.61 

.98 

Hands-on 

10 

175 

116.55 

77.41 

1.50 

Not*.  Abbreviations  are:  HYP,  Hypothesis;  and  MSE,  Mean  square  error. 
*p  <  .05. 

**p  <  .01. 
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Table  G-5.  Aerospace  Ground  Equipment  Mechanics  (AFS  423X5):  Summary  of 
Roy-Bargman  Step-Down  Tests  (Uncorrected  for  Restriction) 


Analysis  df  Mean  Squares 


Set 

Order 

HYP 

MSE 

HYP 

MSE 

F-ratio 

1 

TOTWTPT 

10 

204 

2117.63 

461.40 

4.59** 

TGRD 

10 

203 

85.31 

19.56 

4.36** 

2 

Hands-on 

10 

204 

1012.93 

195.01 

5.19** 

Interview 

10 

203 

90.20 

112.41 

.63 

TGRD 

10 

202 

81.10 

19.58 

4.14** 

3 

TGRD 

10 

204 

122.31 

20.29 

6.03** 

TOTWTPT 

10 

203 

1342.98 

444.74 

3.02** 

4 

TGRD 

10 

204 

122.31 

20.29 

6.03** 

Hands-on 

10 

203 

635.78 

187.93 

3.38** 

Interview 

10 

202 

90.64 

112.56 

.80 

5 

TGRD 

10 

204 

122.31 

20.29 

6.03** 

Interview 

10 

203 

368.44 

189.07 

1.95* 

Hands-on 

10 

202 

241.60 

111.88 

2.16* 

Note.  Abbreviations  are:  HYP,  Hypothesis;  and  MSE,  Mean  square  error. 
•p  <  .05. 

**p  <  .01. 


Table  G-6.  Aircrew  Life  Support  Specialists  (AFS  122X0):  Summary  of  Roy-Bargman 
Step-Down  Tests  (Uncorrected  for  Restriction) 


Analysis  df _  Mean  Squares 


Set 

Order 

HYP 

MSE 

HYP 

MSE 

F-ratio 

1 

TOTWTPT 

10 

161 

324.43 

317.21 

1.02 

TGRD 

10 

160 

70.62 

17.83 

3.96** 

2 

Hands-on 

10 

161 

324.43 

317.21 

1.02 

Interview 

10 

160 

97.20 

60.20 

1.61 

TGRD 

10 

159 

69.85 

17.94 

3.89** 

3 

TGRD 

10 

161 

73.71 

17.75 

4.15** 

TOTWTPT 

10 

160 

279.46 

318.52 

.56 

4 

TGRD 

10 

161 

73.71 

17.75 

4.15** 

Hands-on 

10 

160 

279.46 

318.52 

.88 

Interview 

10 

159 

94.90 

60.58 

1.57 

5 

TGRD 

10 

161 

73.71 

17.75 

4.15** 

Interview 

10 

160 

182.63 

90.03 

2.03* 

Hands-on 

10 

159 

96.75 

214.30 

.45 

Note.  Abbreviations  are:  HYP,  Hypothesis;  and  MSE,  Mean  square  error. 
•p  <  .05. 

**p  <  .01. 
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Table  G-7.  Personnel  Specialists  (AFS  732X0):  Summary  of  Roy-Bargman 
Step-Down  Tests  {Uncorrected  for  Restriction) 


Analysis  df _  Mean  Squares 


Set 

Order 

HYP 

MSE 

HYP 

MSE 

F-ratio 

1 

TOTWTPT 

10 

166 

934.31 

390.85 

2.39* 

TGRD 

10 

165 

125.68 

22.42 

5.60** 

2 

Hands-on 

10 

166 

804.72 

391.44 

2.06* 

Interview 

10 

165 

293.85 

228.30 

1.29 

TGRD 

10 

164 

118.94 

21.44 

5.55** 

3 

TGRD 

10 

166 

146.53 

22.37 

6.65** 

TOTWTPT 

10 

165 

639.98 

391.70 

1.63 

4 

TGRD 

10 

166 

146.53 

22.37 

6.55** 

Hands-on 

10 

165 

371.68 

380.18 

.98 

Interview 

10 

164 

360.26 

225.32 

1.60 

5 

TGRD 

10 

166 

146.53 

22.37 

6.65** 

Interview 

10 

165 

427.46 

239.95 

1.78 

Hands-on 

10 

164 

288.27 

357.00 

.81 

Note.  Abbreviations  are:  HYP,  Hypothesis;  and  MSE.  Mean  square  error. 
*p  <  .05. 

**p  <  .01. 


Table  G-8.  Precision  Measurement  Equipment  Laboratory  Specialists  (AFS  324X0): 
Summary  of  Roy-Bargman  Step-Down  Tests  (Uncorrected  for  Restriction) 


Analysis 

df 

Mean 

Squares 

F-ratio 

Set 

Order 

HYP 

MSE 

HYP 

MSE 

1 

TOTWTPT 

10 

115 

1132.38 

289.68 

3.91* 

TGRD 

10 

114 

104.06 

17.51 

5.94* 

2 

Hands-on 

10 

115 

1071.48 

282.80 

3. 79* 

Interview 

10 

114 

39.39 

25. 3U 

1.56 

TGRD 

10 

113 

99.26 

17.51 

5.67* 

3 

TGRD 

10 

115 

133.69 

18.80 

7.11* 

TOTWTPT 

10 

114 

798.24 

269.85 

2.96* 

4 

TGRD 

10 

115 

133.69 

18.80 

7.11* 

Hands-on 

10 

114 

758.32 

263.74 

2.88* 

Interview 

10 

113 

34.12 

25.28 

1.35 

5 

TGRD 

10 

115 

133.69 

18.80 

7.11* 

Interview 

10 

114 

45.56 

28.33 

1.61 

Hands-on 

10 

113 

607.71 

235.32 

2.58* 

Not*.  Abbreviations  are: 

HYP, 

Hypothesis; 

and  MSE,  Mean  square  error. 

*p  <  .01. 
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APPENDIX  H:  ROY-BARGMAN  STEP-DOWN  TESTS 


FOR  RESTRICTION) 
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Table  H-1.  Air  Traffic  Control  Operators  (AFS  272X0):  Summary  of  Roy-Bargman 
Step-Down  Tests  (Corrected  for  Restriction) 


Analysis  df  Mean  Squares 


Set 

Order 

HYP 

MSE 

HYP 

MSE 

F-ratio 

1 

TOTWTPT 

10 

146 

784.60 

61.40 

2.17* 

TGRD 

10 

145 

563.52 

28.67 

19.65** 

2 

Hands-on 

10 

146 

346. 04 

163.58 

2.12* 

Interview 

10 

145 

275.09 

73.03 

3.77** 

TGRD 

10 

144 

520.18 

28.84 

18.04** 

3 

TGRD 

10 

146 

600.13 

28.53 

21.03** 

TOTWTPT 

10 

145 

572.28 

363.12 

1.58 

4 

TGRD 

10 

146 

600.13 

28.53 

21.03** 

Hands-on 

10 

145 

334.89 

164.62 

2.03* 

Interview 

10 

144 

176.79 

73.35 

2.41* 

5 

TGRD 

10 

146 

600.13 

28.53 

21.03** 

Interview 

10 

145 

292.29 

213.63 

1.37 

Hands-on 

10 

144 

176.12 

56.52 

3.12* 

Note.  Statistical  significance  reported  for  uncorrected  step-down  tests  is  inferred  for  corrected  tests. 
Abbreviations  are:  HYP,  Hypothesis;  and  MSE,  Mean  square  error. 

•p  <  .05. 

**p  <  .01 . 


Table  H-2.  Avionic  Communications  Specialists  (AFS  328X0):  Summary  of  Roy- 
Bargman  Step-Down  Tests  (Corrected  for  Restriction) 


Analysis 

df 

Mean 

Squares 

F-ratio 

Set 

Order 

HYP 

MSE 

HYP 

MSE 

1 

TOTWTPT 

10 

63 

3410.08 

328.52 

10.38** 

TGRD 

10 

62 

319.09 

15.14 

21.07** 

2 

Hands-on 

10 

63 

1143.58 

165.11 

6.92** 

Interview 

10 

62 

426.97 

133.45 

3.20** 

TGRD 

10 

61 

337.00 

15.08 

22.34** 

3 

TGRD 

10 

63 

538.24 

15.30 

35.17** 

TOTWTPT 

10 

62 

1550.12 

325.10 

4.77** 

4 

TGRD 

10 

63 

538.24 

15.20 

35.17** 

Hands-on 

10 

62 

494.24 

165.57 

2.98** 

Interview 

10 

61 

417.31 

131.20 

3.18** 

5 

TGRD 

10 

63 

538.24 

15.30 

35.17** 

Interview 

10 

62 

1003.73 

187.48 

5.35** 

Hands-on 

10 

61 

148.07 

115.87 

1.28 

Note.  Statistical  significance  reported  for  uncorrected  step-down  tests  is  inferred  for  corrected  tests. 
Abbreviations  are:  HYP,  Hypothesis;  and  MSE,  Mean  square  error. 

• p  <  .05. 

**p  <  .01. 
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Table  H-3.  Information  Systems  Radio  Operators  (AFS  492X1):  Summary  of  Roy- 
Bargman  Step-Down  Tests  (Corrected  for  Restriction) 


Analysis  df  Mean  Squares 


Set 

Order 

HYP 

MSE 

HYP 

MSE 

F-ratio 

1 

TOTWTPT 

10 

86 

2224.04 

736.49 

3.02** 

TGRD 

10 

85 

439.56 

26.92 

16.33** 

2 

Hands-on 

10 

86 

1014.96 

430.49 

2.36* 

Interview 

10 

85 

608.27 

121.97 

4.99** 

TGRD 

10 

84 

381.53 

27.09 

14.08** 

3 

TGRD 

10 

86 

547.98 

30.45 

17.99** 

TOTWTPT 

10 

85 

1528.96 

651.01 

2.35* 

4 

TGRD 

10 

86 

547.98 

30.45 

17.99* 

Hands-on 

10 

85 

881.61 

391.54 

2.25* 

Interview 

10 

84 

384.32 

119.28 

3.32** 

5 

TGRD 

10 

86 

547.98 

30.45 

17.99* 

Interview 

10 

85 

870.39 

265.37 

3.28 

Hands-on 

10 

84 

388.52 

175.98 

2.20 

Note.  Statistical  significance  reported  for  uncorrected  step-down  tests  is  inferred  for  corrected  tests 
Abbreviations  sire:  HYP,  Hypothesis;  and  MSE,  Mean  square  error. 

*p  <  .05. 

**p  <  .01. 


Table  H-4.  Jet  Engine  Mechanics  (AFS  426X2):  Summary  of  Roy-Bargman  Step- 
Down  Tests  (Corrected  for  Restriction) 


Analysis  df _  Mean  Squares 


Set 

Order 

HYP 

MSE 

HYP 

MSE 

F-ratio 

1 

TOTWTPT 

10 

155 

964.81 

217.30 

4.44** 

TGRD 

10 

154 

794.53 

28.88 

27.52** 

2 

Hands-on 

10 

155 

350.50 

105.38 

3.33** 

Interview 

10 

154 

299.56 

111.49 

2.69** 

TGRD 

10 

153 

775.59 

28.64 

27.08** 

3 

TGRD 

10 

155 

1021.60 

29.82 

34.26** 

TOTWTPT 

10 

154 

496.61 

202.73 

2.45** 

4 

TGRD 

10 

155 

1021.60 

29.82 

34.26** 

Hands-on 

10 

154 

202.47 

100.69 

2.01* 

Interview 

10 

153 

151.87 

112.09 

1.35 

5 

TGRD 

10 

155 

1021.60 

29.82 

34.26** 

Interview 

10 

154 

234.82 

141.56 

1.66 

Hands-on 

10 

153 

135.42 

79.73 

1.70 

Note  Statistical  significance  reported  for  uncorrected  step-down  tests  is  inferred  for  corrected  tests. 
Abbreviations  are:  HYP,  Hypothesis;  and  MSE,  Mean  square  error. 

•p  <  .05. 

**p  <  .01. 
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Table  H-5.  Aerospace  Ground  Equipment  Mechanics  (AFS  423X5):  Summary  of 
Roy-Bargman  Step-Down  Tests  (Corrected  for  Restriction) 


Analysis  df  Mean  Squares 


Set 

Order 

HYP 

MSE 

HYP 

MSE 

F-ratio 

1 

TOTWTPT 

10 

191 

4747.90 

479.49 

9.90** 

TGRD 

10 

190 

319.64 

20.29 

15.75** 

2 

Hands-on 

10 

191 

2526.99 

199.03 

12.70** 

Interview 

10 

190 

119.50 

112.88 

1.06 

TGRD 

10 

189 

288.86 

20.35 

14.20** 

3 

TGRD 

10 

191 

509.89 

21.04 

24.24** 

TOTWTPT 

10 

190 

2009.31 

460.32 

4.36** 

4 

TGRD 

10 

191 

509.89 

21.04 

24.24** 

Hands-on 

10 

190 

1029.72 

192.25 

5.36** 

Interview 

10 

189 

124.23 

113.10 

1.10 

5 

TGRD 

10 

191 

509.89 

21.04 

24.24** 

Interview 

10 

190 

508.84 

193.56 

2.63* 

Hands-on 

10 

189 

406.48 

112.26 

3.62* 

Note.  Statistical  significance  reported  for  uncorrected  step-down  tests  is  inferred  for  corrected  tests. 
Abbreviations  are:  HYP.  Hypothesis;  and  MSE,  Mean  square  error. 

* p  <  .05. 

**p  <  .01. 


Table  H-6.  Aircrew  Life  Support  Specialists  (AFS  122X0):  Summary  of  Roy- 
Bargman  Step-Down  Tests  (Corrected  for  Restriction) 


Analysis 

df 

Mean 

Squares 

F-ratio 

Set 

Order 

HYP 

MSE 

HYP 

MSE 

1 

TOTWTPT 

10 

146 

280.71 

307.51 

.91 

TGRD 

10 

145 

252.86 

16.94 

14.93** 

2 

Hands-on 

10 

146 

280.71 

307.51 

.91 

Interview 

10 

145 

88.37 

60.36 

1.46 

TGRD 

10 

144 

252.81 

17.05 

14.83** 

3 

TGRD 

10 

146 

263.92 

16.87 

15.64** 

TOTWTPT 

10 

145 

183.72 

308.72 

.59 

4 

TGRD 

10 

146 

263.92 

16.87 

15.64** 

Hands-on 

10 

145 

183.72 

308.72 

.60 

Interview 

10 

144 

88.39 

60.76 

1.45 

5 

TGRD 

10 

146 

263.92 

16.87 

15.64** 

Interview 

10 

145 

143.41 

91.26 

1.57* 

Hands-on 

10 

144 

101.04 

205.54 

.49 

Note  Statistical  significance  reported  for  uncorrected  step-down  tests  is  inferred  for  corrected  tests. 


Abbreviations  are  HYP,  Hypothesis;  and  MSE,  Mean  square  error. 
*p  <  05. 

**p  <  01 . 
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Table  H-7.  Personnel  Specialists  (AFS  732X0):  Summary  of  Roy-Bargman 
Step-Down  Tests  (Corrected  for  Restriction) 


Analysis  df  Mean  Squares 


Set 

Order 

HYP 

MSE 

HYP 

MSE 

F-ratio 

1 

TOTWTPT 

10 

156 

3213.78 

405.02 

7.93* 

TGRD 

10 

155 

275.96 

22.44 

12.30** 

2 

Hands-on 

10 

156 

1889.52 

402.84 

4.69* 

Interview 

10 

155 

1096.88 

230.11 

4.77 

TGRD 

10 

154 

268.95 

21.48 

12.52** 

3 

TGRD 

10 

156 

418.23 

22.35 

18.71** 

TOTWTPT 

10 

155 

1450.45 

406.63 

3.57 

4 

TGRD 

10 

156 

418.23 

22.35 

18.71** 

Hands-on 

10 

155 

490.54 

393.60 

1.25 

Interview 

10 

154 

1037. 1C 

226.32 

4.58 

5 

TGRD 

10 

156 

418.23 

22.35 

18.71** 

Interview 

10 

155 

1272.21 

240.75 

5.28 

Hands-on 

10 

154 

259.12 

370.02 

.70 

Note.  Statistical  significance  reported  for  uncorrected  step-down  tests  is  inferred  for  corrected  tests. 
Abbreviations  are:  HYP,  Hypothesis;  and  MSE,  Mean  square  error. 

*p  <  .05, 

**p  <  .01. 


Table  H-8.  Precision  Measurement  Equipment  Laboratory  Specialists  {AFS  324X0): 
Summary  of  Roy-Bargman  Step-Down  Tests  (Corrected  for  Restriction) 


Analysis  df _  Mean  Squares 


Set 

Order 

HYP 

MSE 

HYP 

MSE 

F-ratio 

1 

TOTWTPT 

10 

112 

5521.42 

297.05 

i  8.59* 

TGRD 

10 

111 

305.82 

16.86 

18.13* 

2 

Hands-on 

10 

112 

4964.40 

289.94 

17.12* 

Interview 

10 

111 

125.38 

25.09 

5.00 

TGRD 

10 

110 

277.74 

16.65 

13.68* 

3 

TGRD 

10 

112 

727.66 

18.18 

40.02* 

TOTWTPT 

10 

111 

1625.69 

275.56 

5.90* 

4 

TGRD 

10 

112 

727.66 

18.18 

40.02* 

Hands-on 

10 

111 

1548.51 

269.18 

5.75* 

Interview 

10 

110 

50.32 

24.75 

2.03 

5 

TGRD 

10 

112 

727.56 

18.18 

40.02* 

Interview 

10 

111 

82.85 

27.66 

3.00 

Hands-on 

10 

110 

1103.71 

240.83 

4.58* 

Note.  Statistical  significance  reported  for  uncorrected  step-down  tests  is  inferred  for  corrected  tests 
Abbreviations  are:  HYP,  Hypothesis;  and  MSE,  Mean  square  error. 

*p  <  01 
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